Motion Based Account Recognition

ABSTRACT

A user establishes an account with an account management system and downloads a service application on a user computing device associated with the user. The user enters a service provider location and signs into a service application. A service camera device captures a video feed of one or more users within a visual field of the service camera device and a service computing device compares movement data received from user computing devices at the service provider location against movement of detected objects in the video feed to identify one or more users in the video feed. After identifying the one or more users within the video feed of the service camera device, in response to a particular user initiating a service request at the service computing device, the service computing device identifies the particular user at the service computing device as being within a service area of the video feed.

TECHNICAL FIELD

The present disclosure relates to identifying user accounts for use inservice requests based on facial recognition of users and correlatingmotion data received from user computing devices to a video feed.

BACKGROUND

When users are provided with services at a service provider location,many methods of conducting service requests are available. Consumers mayuse many different cards or accounts. The user account identifiers andother data represented by the cards may be communicated to the serviceprovider system via magnetic stripes, near field communicationtechnologies involving user computing devices, and other suitablemechanisms.

Current applications for conducting service requests at a serviceprovider system location do not provide the opportunity for the user tomake a hands-free service request and instead require the user tointeract with a user computing device. Additionally, currentapplications require the user to perform actions to identify himself orherself by providing user account identifiers or other data to theservice provider system.

SUMMARY

Techniques herein provide computer-implemented methods to conduct ahands-free service request with facial recognition and movementrecognition. In an example, a service provider system registers with anaccount management system. A user establishes an account with theaccount management system and transmits an image of himself to theaccount management system to establish a facial template associated withthe user account. A user signs into the service application via the usercomputing device and enters the service provider system location. Theuser computing device receives a service beacon device identifier fromthe service beacon device and transmits the identifier to the accountmanagement system. The account management system transmits to theservice computing device facial templates corresponding to other userswhose user computing devices are within network range of the servicebeacon device and who are signed in to the service application. Theservice camera device captures a facial image of the user and theservice computing device identifies the user by comparing the facialimage captured by the service camera device against the received facialtemplates. The service computing device also compares movement datareceived from the account management system for user computing devicesat the service provider system location against movement of usersrecorded by the service camera device to identify one or more users inthe visual field of the service camera device. After identifying the oneor more users within the visual field of the service camera device, inresponse to a particular user initiating a service request at theservice computing device, the service computing device identifies theparticular user based on a proximity to a designated service area in thevideo feed. The service computing device operator selects an account ofthe identified user for use in a service request from one or moredisplayed accounts of the user. The service provider device transmitsservice request details to the account management system, whichgenerates a service request authorization request.

In certain other example aspects described herein, systems and computerprogram products to conduct a hands-free service request with facialrecognition and movement recognition are provided.

These and other aspects, objects, features, and advantages of theexamples will become apparent to those having ordinary skill in the artupon consideration of the following detailed description of illustratedexamples.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram depicting a system for conducting a hands-freeservice request with facial recognition and movement recognition ofusers, in accordance with certain examples.

FIG. 2 is a block flow diagram depicting a method for conducting ahands-free service request with facial recognition and movementrecognition of users, in accordance with certain examples.

FIG. 3 is a block flow diagram depicting a method for registering, by aservice provider system, with an account management system andinstalling hardware at a service provider system location, in accordancewith certain examples.

FIG. 4 is a block flow diagram depicting a method for registering, by auser, for an account with an account management system, in accordancewith certain examples.

FIG. 5 is a block flow diagram depicting a method for establishing afacial template associated with a user account, in accordance withcertain examples.

FIG. 6 is a block flow diagram depicting a method for receiving, by auser computing device, a service beacon device identifier broadcast by aservice beacon device, in accordance with certain examples.

FIG. 7 is a block flow diagram depicting a method for receiving, by acamera device, a facial template for each user in range of a servicebeacon device, in accordance with certain examples.

FIG. 8 is a block flow diagram depicting a method for receiving, by aservice computing device, notification from an account management systemas user devices enter or leave a network range of a service beacondevice, in accordance with certain examples.

FIG. 9 is a block flow diagram depicting a method for identifying, by aservice computing device, a user via facial recognition, in accordancewith certain examples.

FIG. 10 is a block flow diagram depicting a method for identifying, by aservice computing device, a user via movement recognition, in accordancewith certain examples.

FIG. 11 is a block flow diagram depicting a method for initiating, by auser, a service request at a service computing device, in accordancewith certain examples.

FIG. 12 is a block flow diagram depicting a method for processing, by anaccount management system, a service request, in accordance with certainexamples.

FIG. 13 is a block diagram depicting a computing machine and module, inaccordance with certain examples.

DETAILED DESCRIPTION OF EXAMPLES Overview

The examples described herein provide computer-implemented techniquesfor conducting a hands-free service request with facial recognition andmovement recognition.

In an example, a service provider system registers with an accountmanagement system. The service provider system installs one or moreservice beacon devices, one or more service computing devices, and oneor more service camera devices at a service provider system location. Auser establishes an account with the account management system anddownloads a service application on a user computing device associatedwith the user. In an example, the user transmits an image of himself tothe account management system to establish a facial template associatedwith the user account. The user enters a service provider systemlocation and signs into the service application via the user computingdevice. The user computing device receives a service beacon deviceidentifier from the service beacon device and transmits the identifierto the account management system. The account management systemtransmits to the service computing device facial templates correspondingto users whose user computing devices are within network range of theservice beacon device and who are signed in to the service application.The service camera device captures facial images of one or more userswithin a visual field of the service camera device. The servicecomputing device identifies the one or more users based on comparing thefacial images captured by the service camera device against the receivedfacial templates. The service camera device also compares movement datareceived from the account management system for user computing devicesat the service provider system location against movement of usersrecorded by the service camera device to identify one or more users inthe visual field of the service camera device. After identifying the oneor more users within the visual field of the service camera device andin response to a particular user initiating a service request at theservice computing device, the service computing device identifies theparticular user at the service computing device. In an example, theservice computing device displays one or more accounts of the identifiedparticular user. The service computing device operator selects anaccount of the identified particular user for use in a service request.The service computing device transmits service request details to theaccount management system, which generates a service requestauthorization request to transmit to an issuer system associated withthe user account selected for use in the service request. The accountmanagement system receives an approval of the service requestauthorization request and transmits a receipt to the service computingdevice.

In an example, the service computing device displays one or moreaccounts of the identified particular user. The service computing deviceoperator selects an account of the identified particular user for use ina service request. The service computing device transmits servicerequest details to the account management system, which generates aservice request authorization request to transmit to an issuer systemassociated with the user account selected for use in the servicerequest. The account management system receives an approval of theservice request authorization request and transmits a receipt to theservice computing device.

In an example, a service provider system registers with an accountmanagement system. A service computing device operator installs aservice application on a service computing device. In another example,the service computing device operator installs the service applicationon a plurality of service computing devices at a service provider systemlocation. A service beacon device receives a beacon identifier code froman account management system. The service beacon device broadcasts thebeacon identifier code via wireless communication at the serviceprovider system location. The service computing device operator installsa service camera device at the service provider system location tocorrespond to the service computing device. The account managementsystem receives a service computing device identifier corresponding to aservice computing device and associates it with a corresponding beaconidentifier of the service beacon device.

In an example, the user registers with an account management system. Forexample, the user accesses an account management system website via auser computing device associated with the user. The user registers withthe account management system and downloads a service application ontothe user computing device In an example, the account management systemestablishes a facial template associated with the user account. Theaccount management system creates a facial template associated with theuser account based on the received facial image. The account managementsystem deletes the received facial image.

In an example, the user signs in to a service application on the usercomputing device. The user carries the user computing device within athreshold distance of a service beacon device at the service providersystem location. The user computing device receives a service beacondevice identifier broadcast by the service beacon device and transmitsthe received service beacon device identifier and a user accountidentifier to the account management system. The account managementsystem receives the service beacon device identifier and the useraccount identifier. The account management system extracts a facialtemplate associated with the user account identifier and identifies aservice camera device associated with the service beacon deviceidentifier. The account management system transmits a facial template ofthe identified user to the service computing device associated with theservice beacon device identifier. For example, a facial templateassociated with the identified user's account is transmitted to theservice computing device. The service computing device receives thefacial template of the user. The service computing device adds thefacial template of the user to a current customer log. The servicecomputing device periodically updates the current customer log based onupdates received from the account management system. The servicecomputing device captures video of the user via the service cameradevice. The service computing device receives the video feed of theservice camera device and extracts a facial image of the user from thecaptured video and generates a facial template from the captured facialimage. The service computing device deletes the received video feed andextracted facial image. The service computing device retrieves facialtemplates from the current customer log. The service camera devicecompares the generated facial template from the extracted facial imageto facial templates from the current customer log. The service cameradevice is able to identify the user if there is a match between a facialtemplate from the current customer log and the generated facialtemplate. The service camera device is unable to identify the user ifthere is no match between a facial template from the current customerlog and the generated facial template. In an example, if the servicecamera device is able to identify the user, the service camera devicenotifies the account management system of the identity of the user Inanother example, if the service camera device is able to identify theuser, the service camera device notifies the service computing device ofthe identity of the user. In an example, the service computing deviceidentifies one or more users from the video feed based on facialrecognition.

In an example, instead of or in addition to using facial recognition,the service computing device compares the received video feed withmovement data of user computing devices received from the accountmanagement system to identify users in the video feed. In otherexamples, the account management system compares the video feed with themovement data to identify users in the video feed. For example, inaddition to transmitting a beacon device identifier to the accountmanagement system, each user computing device at the service systemlocation transmits movement data logged by the user computing device. Inan example, each user computing device continuously or periodicallytransmits logged movement data to the service computing device via awireless communication channel if the user is signed in to the serviceapplication. In an example, if the user is not signed in to the serviceapplication, the user computing device does not transmit logged movementdata to the service computing device via the wireless communicationchannel. In other examples, the user computing device continuously orperiodically transmits logged movement data to the account managementsystem via the network when the user is signed in to the serviceapplication on the user computing device. Example movement datacomprises accelerometer data and/or gyroscopic data of the usercomputing device, a user computing device identifier, and timestampdata. For example, movement data comprises a velocity of the usercomputing device, an acceleration of the user computing device, anorientation of the user computing device, a rotation rate of the usercomputing device, or other data that describes or indicates anorientation, position, or a movement of the user computing device. Forexample, timestamp data comprises a current time comprising a year,month, date, hour of the day, minute of the hour, second of the minute,millisecond of the second, and/or other appropriate indication orindications of the time at which the timestamp is generated In anexample, the service computing device receives the movement data fromuser computing devices at the service provider location via the wirelesscommunication channel. The service computing device compares a receivedvideo feed with received movement data of user computing devices toidentify users in the video feed. In an example, the service computingdevice receives a continuous video feed from the service computingdevice comprising video captured of an area of the service providersystem location Example video feed data comprises timestamp data. In anexample, the service computing device defines a location of the areacaptured by the video feed associated with a service request area anddetermines an identity of the user computing device within the servicerequest area based on a received user computing device identifier. In anexample, the video feed comprises multiple service request areas, eachservice request area associated with a particular service computingdevice.

In an example, the account management system receives the movement datafrom each user computing device at the service provider system locationvia the network In this example, the account management system receivesa video feed from the service computing device and compares the receivedvideo feed with the movement data of user computing devices to identifyusers in the video feed. In this example, the account management systemtransmits identities of the users identified in the video feed to theservice computing device. In another example, the account managementsystem transmits the received movement data to the service computingdevice at the service provider location. In this other example, theservice computing device compares the received video feed with movementdata of user computing devices received from the account managementsystem to identify users in the video feed. In an example, the servicecomputing device defines a location of the area captured by the videofeed associated with a service request area and determines an identityof the user within the service request area. In an example, the videofeed comprises multiple service request areas, each service request areaassociated with a particular service computing device. In this example,the service computing device or account management system determines anidentity of a particular user within each service request areaassociated with each particular corresponding service computing device.

In an example, the service computing device or the account managementsystem may be trained to identify users in a video feed based onmovement data received from user computing devices. For example, aservice application on each user computing device at the serviceprovider system location logs data from accelerometers and/or gyroscopesand sends that data via a wireless communication channel to a processingunit of the service computing device. The processing simultaneouslyrecords a video feed from a service camera device communicativelycoupled to the service computing device. In other examples, the servicecomputing device comprises the service camera device. In an example, theprocessing unit uses a machine learning model to identify similaritiesbetween gyroscopic and accelerometer data and visual data from the videoreed received from the service camera device. In an example, the machinelearning model identifies similarities between gyroscopic andaccelerometer data and visual data from the video feed that correspondsto the same time period as the received gyroscopic and accelerometerdata. In this example, the time period may be determined based on timestamp data associated with the received video feed data and the receivedgyroscopic and accelerometer data for each user computing device. Forexample, if a user were to pull his or her user computing device out ofhis or her pocket at a time period A, the processing unit would identifya signature both in the video feed as well as a corresponding signaturefrom the received movement data comprising the gyroscopic and theaccelerometer data during the same time period A. The processing unitwould associate the changes in the movement data with changes in thevideo feed data during the time period and would associate a usercomputing device identifier with a service request area in the videofeed for the time period. In an example, the time period comprises atime period that terminates at the current time. In an example, theservice request area corresponds to a location in front of the servicecomputing device. The processing unit would identify the user based onthe user computing device identifier or receive, from the accountmanagement system, an identity of the user in response to transmitting arequest for the identity of the user to the account management systemalong with the user computing de % ice identifier in the service requestarea.

For example, the machine learning system is structured as follows:received gyroscopic data and accelerometer data for each user computingdevice at the location is entered into a Long Short Term Memory (“LSTM”)Model and a Deep Neural Network (“DNN”) classifier is used to comparedata from the video feed to the gyroscopic data and accelerometer datato detect users within the video feed. In an example, the DNN classifieris able to detect potential users within the video feed based on objectrecognition. For example, the DNN classifier executes a person detectionalgorithm on the service computing device to determine where each personin each frame of the video feed is located In an example, the DNNClassifier determines whether a region of the video feed thatcorresponds to a potential user is associated with a particular usercomputing device represented by the received user computing devicemovement data. In an example, the LSTM Model would be running on theservice application on each user computing device at the serviceprovider system location to generate a serializable encoding ofgyroscopic signatures and accelerometer signatures that the DNNClassifier can compare with the visual data for each detected person inthe video feed to determine a similarity score. In an example, if thesimilarity score for a particular user computing device identifier isgreater than a predetermined threshold, the DNN classifier determinesthat the user computing device identifier corresponds to the detectedperson in the video feed.

In other examples, instead of determining the user computing deviceidentifier that corresponds to the detected person in the video feedbased on a similarity score being greater than a predeterminedthreshold, the DNN classifier uses a tiered threshold matching system Inan example tiered threshold matching system, the DNN determines a strongmatch and a weak match for each detected person in the video feed. Theexample strong match comprises a higher threshold similarity score thanthe example weak match so as to maximize precision. The example weakmatch comprises a threshold similarity score low enough so that anactual present match would be a weak match. In an example, the DNNclassifier generates strong and weak matches for all detected objectsdetected to be persons in the video feed and tracks a unique strongmatch for a detected object or, if a strong match is not present, aunique weak match for the detected object for all frames of the videofeed. Once all detected objects resembling potential persons aretracked, the DNN classifier determines if multiple users are determinedto match a single tracked object in any frame of the video feed. Anyframes of the video feed for the single tracked object for which thetracked object is assigned to more than one user computing device areexcluded from consideration when determining movement of the object inthe video feed. In an example, the LSTM Model would be running on theservice application on each user computing device at the serviceprovider system location to generate a serializable encoding ofgyroscopic signatures and accelerometer signatures that the DNNClassifier can compare with the visual data for each tracked object inthe video feed, excluding any frames of the video feed for which thetracked object was assigned to multiple user computing devices, todetermine a similarity score. In an example, if the similarity score fora particular user computing device identifier is greater than apredetermined threshold, the DNN classifier determines that the usercomputing device identifier corresponds to the detected person in thevideo feed.

The user approaches a service computing device. The service computingdevice operator totals items of the user for purchase. The servicecomputing device operator asks the user to select a processing option.The user directs the service computing device operator to initiate aservice request via the service application. The service computingdevice operator selects an option on the service computing device toinitiate a service request using the service application. The servicecomputing device identifies the user at the service computing devicethat is initiating the service request. For example, the servicecomputing device identifies a user within the service request areaassociated with the service computing device captured by the video feedbased on movement data and/or facial recognition. The service computingdevice transmits a request for account information and an identity ofthe identified user within the service request area of the video feed tothe account management system via the network. The account managementsystem receives the request for account information and the identity ofthe identified user within the service request area of the video feed.In other examples, the service computing device already comprisesaccount information of the identified user.

The service computing device displays accounts of the identified user.The service computing device operator selects a user account for theservice request and confirms the service request with permission of theuser. The service computing device transmits service request details tothe account management system. For example, the service request detailscomprise a total amount, a selected user account for use in the servicerequest, an account of the service provider for use in the servicerequest, and other useful or relevant information. The accountmanagement system transmits a service request authorization request toan issuer system. For example, the issuer system is associated with theuser account selected for use in the service request. The issuer systemapproves the service request authorization request and transmits aservice request authorization approval to the account management system.The account management system transmits a service request receipt to theservice computing device.

By using and relying on the methods and systems described herein, theaccount management system, the service camera device, the service beacondevice, the user computing device, and the service computing deviceenable the user to conduct a service request with the service providersystem without the user having to interact with the user computingdevice or produce identity documents or physical account cards, asrequired in some current technology. As such, the systems and methodsdescribed herein may reduce the inputs required by the user via the usercomputing device and the inputs required by the service computing deviceoperator to identify the user. Further, by using and relying on themethods and systems described herein, the account management system, theservice camera device, the service beacon device, the user computingdevice, and the service computing device enable the user to conduct aservice request with the service provider system without the servicecomputing device having to rely entirely on facial recognition torecognize users. By correlating movement data received from a usercomputing devices to an object in a video feed identified as users, aservice computing device may identify a user in cases where the videofeed captured images of the user over a period of time but was unable tocapture a facial image of the user that would enable identification viafacial recognition.

Example System Architecture

Turning now to the drawings, in which like numerals indicate like (butnot necessarily identical) elements throughout the figures, examples aredescribed in detail.

FIG. 1 is a block diagram depicting a system 100 for conducting ahands-free transaction with facial recognition and movement recognitionof users 101, in accordance with certain examples. As depicted in FIG.1, the system 100 includes network computing devices 110, 130, 140, and150 that are configured to communicate with one another via one or morenetworks 120. In some embodiments, a user associated with a device mustinstall an application and/or make a feature selection to obtain thebenefits of the techniques described herein.

In examples, the network 140 can include a local area network (“LAN”), awide area network (“WAN”), an intranet, an Internet, storage areanetwork (“SAN”), personal area network (“PAN”), a metropolitan areanetwork (“MAN”), a wireless local area network (“WLAN”), a virtualprivate network (“VPN”), a cellular or other mobile communicationnetwork, Bluetooth, Bluetooth low energy, NFC, or any combinationthereof or any other appropriate architecture or system that facilitatesthe communication of signals, data, and/or messages. Throughout thediscussion of examples, it should be understood that the terms “data”and “information” are used interchangeably herein to refer to text,images, audio, video, or any other form of information that can exist ina computer-based environment.

Each network computing device 110, 130, 140, and 150 includes a devicehaving a communication module capable of transmitting and receiving dataover the network 140. For example, each network computing device 110,130, 140, and 150 can include a server, desktop computer, laptopcomputer, tablet computer, a television with one or more processorsembedded therein and/or coupled thereto, smart phone, handheld computer,personal digital assistant (“PDA”), or any other wired or wireless,processor-driven device. In the example depicted in FIG. 1, the networkcomputing devices 110, 130, 140, 150, and 160 are operated by users 101,service beacon device 120 operators, service provider point of sale(“POS”) device 130 operators, service camera device 135 operators andaccount management system 150, respectively.

An example user computing device 110 comprises a user interface 111, aservice application 112, an antenna 113, a Wi-Fi controller 114, a datastorage unit 115, a camera component 116, a web browser 117 a, acommunication application 117 b, an accelerometer 118, and a globalpositioning system (“GPS”) module 119.

In an example, the user interface 111 enables the user 101 to interactwith the service application 112 and/or web browser 117 a. For example,the user interface 111 may be a touch screen, a voice-based interface,or any other interface that allows the user 101 to provide input andreceive output from an application or module on the user computingdevice 110. In an example, the user 101 interacts via the user interface111 with the service application 112 and/or web browser 117 a toconfigure user 101 accounts on the account management system 150. Inanother example, the user 101 interacts via the user interface 111 withthe service application 112 and/or the web browser 117 a to enablehands-free payments, if needed.

In an example, the service application 112 is a program, function,routine, applet, or similar entity that exists on and performs itsoperations on the user computing device 110. In certain examples, theuser 101 must install the service application 112 and/or make a featureselection on the user computing device 110 to obtain the benefits of thetechniques described herein. In an example, the user 101 may access theservice application 112 on the user computing device 110 via the userinterface 111. In an example, the service application 112 may beassociated with the account management system 150. In another example,the service application 112 may be associated with a service providersystem associated with the service beacon device 120, the servicecomputing device 130 and the service camera device 135. In yet anotherexample, two service applications 112 exist, one associated with theservice provider system and another associated with the accountmanagement system 150.

In an example, the antenna 113 is a means of communication between theuser computing device 110 and a service beacon device 120. In anexample, a Wi-Fi controller 114 outputs through the antenna 113 a radiosignal, or listens for radio signals from the service beacon device 120.In another example a Bluetooth controller or a near field communication(“NFC”) controller is used In an example, the Wi-Fi controller 114outputs through the antenna 113 a radio signal, or listens for radiosignals from the service beacon device 120.

In an example, the Wi-Fi controller 114 is capable of sending andreceiving data, performing authentication and ciphering functions, anddirecting how the user computing device 110 will listen fortransmissions from the service beacon device 120 or configuring the usercomputing device 110 into various power-save modes according toWi-Fi-specified procedures. In another example, the user computingdevice 110 comprises a Bluetooth controller or an NFC controller capableof performing similar functions. An example Wi-Fi controller 114communicates with the service application 112 and is capable of sendingand receiving data over a wireless, Wi-Fi communication channel. Inanother example, a Bluetooth controller 112 or NFC controller 112performs similar functions as the Wi-Fi controller 114 using Bluetoothor NFC protocols. In an example, the Wi-Fi controller 114 activates theantenna 113 to create a wireless communication channel between the usercomputing device 110 and the service beacon device 120. The usercomputing device 110 communicates with the service beacon device 120 viathe antenna 113. In an example, when the user computing device 110 hasbeen activated, the Wi-Fi controller 114 polls through the antenna 113 aradio signal, or listens for radio signals from the service beacondevice 120.

In an example, the data storage unit 115 comprises a local or remotedata storage structure accessible to the user computing device 110suitable for storing information. In an example, the data storage unit115 stores encrypted information, such as HTML5 local storage.

In an example, the camera component 116 may be any module or function ofthe user computing device 110 that obtains a digital image. The cameracomponent 116 may be resident on the user computing device 110 or in anymanner logically connected to the user computing device 110. Forexample, the camera component 116 may be connected to the user computingdevice 110 via the network 140. The camera component 116 may be capableof obtaining individual images or a video scan. Any other suitable imagecapturing device may be represented by the camera component 116.

In an example, the user 101 can use a communication application 117 b,such as a web browser 117 a application or a stand-alone application, toview, download, upload, or otherwise access documents or web pages via adistributed network 140.

In an example, the web browser 117 a can enable the user 101 to interactwith web pages using the user computing device 110. In an example, theuser 101 may access the user's 101 account maintained by the accountmanagement system 150 via the web browser 117 a. In another example, theuser 101 may access the service provider system website via the webbrowser 117 a In certain examples described herein, one or morefunctions performed by the service application 112 may also be performedby a web browser 117 a application associated with the accountmanagement system 150.

In an example, the communication application 117 b can interact with webservers or other computing devices connected to the network 140,including the user computing device 110 and a web server of a serviceprovider system.

In certain examples, one or more functions herein described as performedby the service application 112 may also be performed by a web browser117 a application, for example, a web browser 117 a applicationassociated with a service provider system website or associated with theaccount management system 150. In certain examples, one or morefunctions herein described as performed by the service application 112may also be performed by the user computing device 110 operating system.In certain examples, one or more functions herein described as performedvia the web browser 117 a may also be performed via the serviceapplication 112.

In an example, the accelerometer 118 is able to detect a magnitude anddirection of acceleration or magnitude and direction of velocity of theuser computing device 110. In an example, the accelerometer 118communicates with the service application 112 or the service application112 is otherwise able to receive data from the accelerometer 118. In anexample, the accelerometer comprises a gyroscope or the user computingdevice 110 comprises both an accelerometer 118 and a gyroscope. In anexample, the gyroscope is able to detect an orientation of the usercomputing device 110, a rotation rate of the user computing device 110,or other data that describes or indicates an orientation, position, or amovement of the user computing device 110. In an example, the gyroscopecommunicates with the service application 112 or the service application112 is otherwise able to receive data from the gyroscope.

In an example, the GPS module 119 is able to detect a current locationof the user computing device 110 comprising a longitude and a latitudeor other descriptive data describing a location of the user computingdevice 110. In an example, the GPS module 119 communicates with theservice application 112 or the service application 112 is otherwise ableto receive data from the GPS module 119.

An example service beacon device 120 comprises an antenna 121 and aWi-Fi controller 122. In an example, a service provider system locationcomprises one or more service beacon devices 120 installed at theservice provider system location. In an example, each installed servicebeacon device 120 is associated by an account management system 150 witha particular service computing device 140 installed at the serviceprovider location. For example, the account management system 150 maycomprise a database that correlates service beacon device 120identifiers with service computing device 140 identifiers for associatedservice camera devices 140. For example, a service computing device 140identifier may comprise hardware identifier specific to the device suchas a serial number or a MAC ID associated with the service computingdevice 140. In another example, a service beacon device 120 identifiermay comprise a hardware identifier specific to the beacon device or anidentifier generated by the account management system 130 and stored inthe service beacon device 120. An example service beacon device 120 isprogrammed to broadcast, emit, or otherwise transmit a particularservice beacon device 120 identifier over a wireless network 140 to anyuser computing devices 110 within a threshold distance required tomaintain the wireless network 140. For example, the wireless network maycomprise a Wi-Fi network 140, a Bluetooth network 140, an NFC network140, or any other appropriate wireless network 140.

In an example, the antenna 121 is a means of communication between theuser computing device 110 and a service beacon device 120. In anexample, a Wi-Fi controller 122 outputs through the antenna 121 a radiosignal, or listens for radio signals from the user computing device 110.In another example a Bluetooth controller or a near field communication(“NFC”) controller is used. In an example, the Wi-Fi controller 122outputs through the antenna 121 a radio signal, or listens for radiosignals from the payment card device 120.

In an example, the Wi-Fi controller 122 is capable of sending andreceiving data, performing authentication and ciphering functions, anddirecting how service beacon device 120 will listen for transmissionsfrom the user computing device 110 or configuring the service beacondevice 120 into various power-save modes according to Wi-Fi-specifiedprocedures. In another example, the service beacon device 120 comprisesa Bluetooth controller or an NFC controller capable of performingsimilar functions. An example Wi-Fi controller 122 communicates with theservice application 112 and is capable of sending and receiving dataover a wireless, Wi-Fi communication channel. In another example, aBluetooth controller 122 or NFC controller 122 performs similarfunctions as the Wi-Fi controller 122 using Bluetooth or NFC protocols.In an example, the Wi-Fi controller 122 activates the antenna 121 tocreate a wireless communication channel between the user computingdevice 110 and the service beacon device 120. The service beacon device120 communicates with the user computing device 110 via the antenna 121.In an example, when the service beacon device 120 has been activated,the Wi-Fi controller 122 polls through the antenna 121 a radio signal,or listens for radio signals from the user computing device 110.

An example service computing device 130 comprises a user interface 131,a service application 132, a communication application 133, a datastorage unit 134, and a service camera device 135, which comprises acamera component 136, a processor 137, and a data storage unit 139.

In an example, the user interface 131 enables the service provider POSdevice operator 102 to interact with the service computing device 130.For example, the user interface 131 may be a touch screen, a voice-basedinterface, or any other interface that allows the service provider POSdevice operator 102 to provide input and receive output from anapplication or module on the service computing device 130. In anexample, the service provider POS device operator 102 interacts via theuser interface 131 with the service application 132.

In an example, the service application 132 is a program, function,routine, applet, or similar entity that exists on and performs itsoperations on the service computing device 130. In certain examples, theservice provider point of sale (“POS”) device operator 102 or otherservice provider system operator must install the service application132 and/or make a feature selection on the service computing device 130to obtain the benefits of the techniques described herein. In anexample, the service provider POS device operator 102 may access theservice application 132 on the service computing device 130 via the userinterface 131. In an example, the service application 132 may beassociated with the account management system 150. In another example,the service application 132 may be associated with a service providersystem associated with the service beacon device 120 and the servicecamera device 135. In yet another example, two service applications 133exist, one associated with the service provider system and anotherassociated with the account management system 150.

In an example, the communication application 133, such as a web browserapplication or a stand-alone application, enables an operator of theservice computing device 130 to view, download, upload, or otherwiseaccess documents or web pages via a distributed network 140. Forexample, the communication application 133 may enable communication overthe network 140 with the account management system 150. In an example,the communication application 133 enables the service computing device130 and/or service camera device 135 to communicate with a server of theaccount management system 150.

In an example, the data storage unit 134 comprises a local or remotedata storage structure accessible to the service computing device 130suitable for storing information. In an example, the data storage unit134 stores encrypted information, such as HTML5 local storage.

An example service camera device 135 comprises a camera component 136, aprocessor 137, and a data storage unit 139.

In an example, the camera component 136 may be any module or function ofthe service camera device 135 that captures a video input or captures adigital image of an external environment of the service camera device135. The camera component 136 may be resident on the service cameradevice 135 or in any manner logically connected to the service cameradevice 135. For example, the camera component 136 may be connected tothe service camera device 135 via the network 140. The camera component136 may be capable of obtaining individual images or a video scan. Anyother suitable image capturing device may be represented by the cameracomponent 136.

In an example, the processor 137 performs one or more functionsdescribed herein as being performed by the service camera device 135.

In an example, the data storage unit 139 comprises a local or remotedata storage structure accessible to the service camera device 135suitable for storing information. In an example, the data storage unit139 stores encrypted information, such as HTML5 local storage.

An example issuer system (not depicted) approves or denies a paymentauthorization received from the account management system 150. Forexample, a service request may comprise a request to process a paymenttransaction. In an example, the issuer system communicates with theaccount management system 150 over the network 140. In an example, theissuer system communicates with an acquirer system to approve a creditauthorization and to make payment to the account management system 150and/or service provider system. For example, the acquirer system is athird party account management company.

An example account management system 150 comprises an account managementmodule 151, a movement recognition module 152, a facial recognitionmodule 153, a service processing module 154, a data storage unit 155, aserver 157, and a website 159.

In an example, the account management module 151 manages one or moreuser 101 accounts. In an example, a user 101 account may comprise adigital wallet account, an email account, a social networking account,or any other appropriate account associated with the account managementsystem 150. In an example, the account management system 161communicates with a service application 112 operating on a usercomputing device 110 associated with a user 101 having a user 101account with the account management system 150. In an example, the user101 enters payment account information into the user 101 account via theservice application 112 and the account management module 151 receivesthe payment account information over the network 140 and associates thereceived payment account information with the user 101 account.

In an example, the movement recognition module 152 receives the movementdata from each user computing device 110 at the service provider systemlocation via the network 140. In this example, the movement recognitionmodule 152 receives a video feed from the service computing device 130and compares the received video feed with the movement data of usercomputing devices 110 to identify users in the video feed. In thisexample, the movement recognition module 152 transmits identities of theusers identified in the video feed to the service computing device 130.In another example, the movement recognition module 152 transmits thereceived movement data to the service computing device 130 at theservice provider location. In this other example, the service computingdevice 130 compares the received video feed with movement data of usercomputing devices 110 received from the movement recognition module 152to identify users in the video feed. In an example, the servicecomputing device 130 defines a location of the area captured by thevideo feed associated with a service request area and determines anidentity of the user within the service request area. In an example, thevideo feed comprises multiple service request areas, each servicerequest area associated with a particular service computing device 130.In this example, the service computing device 130 or movementrecognition module 152 an identity of a particular user within eachservice request area associated with each particular correspondingservice computing device 130.

In an example, the facial recognition module 153 receives a facial imageof a user 101 associated with a user 101 account submitted by the user101 via the user computing device 110 over the network 140. For example,the user 101 submits the facial image at the time the user 101establishes the user 101 account with the account management system 150.In an example, the facial recognition module 153 generates a facialtemplate based on a received facial image.

In an example, the service processing module 154 receives transactiondetails from a service computing device 130 and a request to initiate atransaction. Example transaction details comprise service providersystem account information, a total amount of the transaction, and auser 101 selection of a user 101 payment account associated with theuser's 101 account with the account management system 150. For example,the user's 101 account is a digital wallet account comprising one ormore payment account information corresponding to one or more respectivepayment accounts of the user 101 in an example, the service processingmodule 154 extracts payment account information from the user 101account corresponding to the user 101 selection of the user 101 paymentaccount received in the transaction details from the service computingdevice 130. In an example, the service processing module 154 transmits apayment authorization request to an issuer system or other appropriatefinancial institution associated with the payment account selected bythe user 101 for use in the transaction. An example paymentauthorization request may comprise service provider system paymentaccount information, user 101 payment account information, and a totalamount of the transaction. In an example, after the issuer systemprocesses the payment authorization request, the service processingmodule 154 receives an approval or denial of the payment authorizationrequest from the issuer system over the network 140. In an example, theservice processing module 154 transmits a receipt to the servicecomputing device 130 and/or the user computing device 110 comprising asummary of the transaction.

In an example, the data storage unit 155 comprises a local or remotedata storage structure accessible to the account management system 150suitable for storing information. In an example, the data storage unit155 stores encrypted information, such as HTML5 local storage.

In an example, the server 157 provides content accessible by the user101 through the web browser 117 a and/or service application 112 on theuser computing device 110, including but not limited to html documents,images, style sheets, and scripts. In an example embodiment, the server157 supports the account management system website 159.

In an example embodiment, the website 159 provides the serviceapplication 112 for download by the user computing device 110 and theservice application 132 for download by the service computing device130. In an example embodiment, the account management system website 159is accessed by the user 101 via the web browser 117 a. In anotherexample embodiment, the user 101 accesses the website 159 via a serviceapplication 112 resident on the user computing device 110.

It will be appreciated that the network connections shown are exampleand other means of establishing a communications link between thecomputers and devices can be used. Moreover, those having ordinary skillin the art having the benefit of the present disclosure will appreciatethat the user computing device 110, the service beacon device 120, theservice computing device 130, the service camera device 135, the issuersystem, and the account management system 150 illustrated in FIG. 1 canhave any of several other suitable computer system configurations. Forexample, a user computing device 110 embodied as a mobile phone orhandheld computer may or may not include all the components describedabove.

In examples, the network computing devices and any other computingmachines associated with the technology presented herein may be any typeof computing machine such as, but not limited to, those discussed inmore detail with respect to FIG. 13. Furthermore, any modules associatedwith any of these computing machines, such as modules described hereinor any other modules (scripts, web content, software, firmware, orhardware) associated with the technology presented herein may by any ofthe modules discussed in more detail with respect to FIG. 13. Thecomputing machines discussed herein may communicate with one another aswell as other computer machines or communication systems over one ormore networks, such as network 140. The network 140 may include any typeof data or communications network, including any of the networktechnology discussed with respect to FIG. 13.

Example Processes

The example methods illustrated in FIGS. 2-12 are described hereinafterwith respect to the components of the example operating environment 100.The example methods of FIGS. 2-12 may also be performed with othersystems and in other environments.

FIG. 2 is a block diagram depicting a method 200 for conducting ahands-free transaction with facial recognition and movement recognitionof users 101, in accordance with certain examples. The method 200 isdescribed with reference to the components illustrated in FIG. 1.

In block 210, the service provider system registers with the accountmanagement system 150 and installs hardware in a service providerlocation. The method for registering, by a service provider system, witha account management system 150 and installing hardware at a serviceprovider system location is described in more detail hereinafter withreference to the method described in FIG. 3.

FIG. 3 is a block diagram depicting a method 210 for registering, by aservice provider system, with an account management system ISO andinstalling hardware at a service provider system location, in accordancewith certain examples. The method 210 is described with reference to thecomponents illustrated in FIG. 1.

In the examples described herein, the service provider system does notneed to install hardware at the example service provider system locationin any particular order. The method 210 describes one example method ofinstalling hardware at the service provider location. However, theservice provider system or other system installing the service providerhardware does not need to install the service computing device 130, theservice camera device 135, or the service beacon device 120 in the orderdescribed herein. Further, in certain examples, the service computingdevice 130 the service camera device 135 and, in certain examples, theservice computing device 130 comprises the service beacon device 120. Inyet other examples, the service computing device 130 comprises theservice camera device 135 and the service beacon device 120.

In block 310, a service provider system registers with the accountmanagement system ISO. In an example, an agent of the service providersystem accesses a account management system 150 website 159 andregisters for a service provider account with the account managementsystem 150 via the website 159. In an example, the service providersystem adds account information associated with a service provideraccount to the service provider account managed by the accountmanagement system 150. For example, the account information comprisespayment account information associated with a service provider systempayment account. In an example, the service provider system comprisesone or more service provider system locations. For example, the serviceprovider system may comprise one or more physical store locations. Anexample service provider location comprises one or more servicecomputing devices 130. In an example, one or more service systemoperators 102 operate the one or more service computing devices 130 atthe service provider system location.

In block 320, a service provider system operator 102 installs theservice application 132 on the service computing device 130. In anotherexample, the service provider system operator purchases a servicecomputing device 130 from the account management system 150 with theservice application 132 pre-installed on the service computing device130. In an example, the service computing device 130 is able tocommunicate with the account management system 150 over a network 140.In an example, the service computing device 130 communicates with theaccount management system 150 via the service application 132. Forexample, the service computing device 130 may be able to transmittransaction details to the account management system 150 via the serviceapplication 132 over the network 140 to enable the account managementsystem 150 to process a transaction. In another example, the servicecomputing device 130 may be able to receive a receipt from the accountmanagement system 150 that notifies a service provider POS deviceoperator 102 as to whether a transaction was successful or not.

In block 330, the service beacon device 120 receives a beacon identifiercode from the account management system 150. In an example, the serviceprovider system receives a beacon identifier from the account managementsystem 150 and installs or otherwise saves the beacon identifier on theservice beacon device 120. In an example, a service provider systemoperator installs the service beacon device 120 in proximity to aservice computing device 130. In an example, the service provider systemoperator installs a plurality of service beacon devices 120, eachservice beacon device 120 in proximity to one or more associated servicecomputing devices 130. In an example, the service beacon device 120 isable to broadcast a service beacon device 120 identifier over a wirelessmedium, wherein one or more user computing devices 110 located within athreshold proximity to the service beacon device 120 are able to receivethe service beacon device 120 identifier over the wireless medium. Inanother example, the service beacon device 120 is able to establish alocal network 140 connection to one or more user computing devices 110located within a threshold proximity to the service beacon device 120and the service beacon device 120 transmits the service beacon device120 identifier to the one or more user computing devices 110 over theestablished local network 140 connection. For example, the thresholdproximity depends on the network 140 communication protocol utilized bythe service beacon device 120.

In block 340, the service beacon device 120 broadcasts the beaconidentifier code via wireless communication at the location of theservice provider system. For example, the service beacon device 120 maybroadcast, emit, or otherwise transmit data comprising the beaconidentifier via Wi-Fi, Bluetooth, Bluetooth low energy (“BLE”), nearfield communication (“NFC”), or other appropriate communication protocolto one or more user computing devices 110 located at the serviceprovider system location within a threshold proximity to the servicebeacon device 120. In some examples, the service beacon device 120, at atime before transmitting the service beacon device 120 identifier, isoperable to establish a network 140 connection between the servicebeacon device 120 and one or more user computing devices 110 located atthe service provider system location within a threshold proximity to theservice beacon device 120.

In block 350, a service provider system operator 102 installs theservice camera device 135 at the service provider system location tocorrespond to the service beacon device 120. In an example, both aservice camera device 135 and a service beacon device 120 are installedin proximity to a particular service computing device 130. In anotherexample, a service camera device 135 and a service beacon device 120 areinstalled in proximity to two or more particular service computingdevices 130. In an example, the service camera device 135 is oriented tobe able to capture video and/or images of a face of a user 101 standingin front of one or more service computing devices 130 during the processof checkout. In an example, the service provider system installs aservice camera device 135 that is oriented to capture video and/orimages of the face of a user standing in front of a particular servicecomputing device 130. In another example, the service provider systeminstalls a service camera device 135 that is oriented to capture videoand/or images of the faces of one or more users 101 standing within aproximity of a particular plurality of service computing devices 130within a range of a field of vision of the camera component 136 of theservice camera device 135.

In block 360, the account management system 150 receives a servicecamera device 135 identifier and associates it with the correspondingbeacon identifier code of the service beacon device 120. In an example,the service provider system and/or the account management system 150configures the service camera device 135 so that the service cameradevice 135 is able to communicate with the account management system 150over the network 140. An example service camera device 135 identifiercomprises a hardware identifier, a MAC address, or other useful orrelevant identifier associated with the service camera device 135. In anexample, the account management system 150 comprises a databasecomprising service camera device 135 identifiers, service computingdevice 130 identifiers, and associated beacon identifiers for servicebeacon device 120 identifiers for a particular service provider systemlocation. In an example, the service camera device 135 transmits theservice beacon device 120 identifier in addition to the service cameradevice 135 identifier to the account management system 150. In anexample, the service camera device 135, during the setup andinstallation process, may receive the service beacon device 120identifier over an appropriate wireless communication channel from theservice beacon device 120. In another example, the service camera device135, during the setup and installation process, may establish a network140 connection with the service beacon device 120 and receive theservice beacon device 120 identifier over the network 140. In anotherexample, the account management system 150 receives the service cameradevice 135 identifier, extracts one or more service beacon device 120identifiers from the database, and associates the service camera device135 identifier with one or more of the one or more extracted servicebeacon device 120 identifiers. In yet another example, the serviceprovider system operator installs the one or more service beacon devices120 after installing the one or more service camera devices 140. In thisexample, the account management system 150 generates a service beacondevice identifier to associate with a service camera device 135identifier and transmits the generated service beacon device identifierto the service provider system. In this example, the service providersystem operator manually configures the service beacon device 120 tobroadcast, emit, or otherwise transmit the service beacon deviceidentifier assigned by the account management system 150 over a network140.

In certain examples, one or both of the service camera device 135 andthe service beacon device 120 are components of the service computingdevice 130 or are wirelessly or physically connected to the servicecomputing device 130 and controlled by one or more processors of theservice computing device 130. In certain examples, certain functionsdescribed herein as performed by the service camera device 135 and/orthe service beacon device 120 may also be performed by the servicecomputing device 130.

From block 360, the method 210 proceeds to block 220 of FIG. 2.

Returning to FIG. 2, in block 220, the user 101 registers with theaccount management system 150. The method for registering, by a user101, for an account with a account management system 150 is described inmore detail hereinafter with reference to the method 220 described inFIG. 4.

FIG. 4 is a block diagram depicting a method 220 for registering, by auser 101, for an account with an account management system 150, inaccordance with certain examples. The method 220 is described withreference to the components illustrated in FIG. 1.

In block 410, the user 101 accesses the account management systemwebsite 159. For example, the user 101 accesses the account managementsystem 150 via the web browser 117 a of the user computing device 110.In another example, the user 101 may otherwise contact the accountmanagement system 150 to register for a user 101 account.

In block 420, the user 101 registers with the account management system150. The user 101 may obtain a user account number, receive theappropriate applications and software to install on the user computingdevice 110, request authorization to participate in hands-free accountmanagement, or perform any action required by the account managementsystem 150. The user 101 may utilize the functions of the user computingdevice 110, such as the user interface 111 and the web browser 117 a, toregister and configure a user 101 account. In an example, the user 101may enter payment account information associated with one or more user101 accounts, for example, one or more credit accounts, one or more bankaccounts, one or more stored value accounts, and/or other appropriateaccounts into the user 101 account maintained by the account managementsystem 150.

In block 430, the user 101 downloads the service application 112 ontothe user computing device 110. In an example, the service application112 operating on the user computing device 110 is able to communicatewith the account management system 150 over the network 140. In anexample, the user 101 may configure user 101 account settings or add,delete, or edit payment account information via the service application112. In an example, the user 101 may select an option to enable ordisable the permission of the account management system 150 to processhands free transactions. For example, a hands free transaction comprisesa transaction wherein the user 101 does not need to interact with theuser computing device 110 or requires minimal user 101 interaction withthe user computing device 110 to initiate a transaction with the serviceprovider system.

In block 440, the account management system 150 establishes a facialtemplate associated with the user 101 account. The method forestablishing a facial template associated with a user 101 account isdescribed in more detail hereinafter with reference to the method 440described in FIG. 5.

FIG. 5 is a block diagram depicting a method 440 for establishing afacial template associated with a user 101 account, in accordance withcertain examples. The method 440 is described with reference to thecomponents illustrated in FIG. 1.

In block 510, the service application 112 displays a request for theuser 101 to capture a facial image via the user computing device 110. Inan example, the service application 112 displays the request via theuser interface 111. In an example, the user interface 111 may display arequest that reads, “to enable hands free service requests, we need animage of your face. Would you like submit a facial image now?” In thisexample, the user 101 may select an option to take a current picture ormay otherwise select a picture stored on the user computing device 110.

In block 520, the user 101 selects an option to capture a facial image.For example, the user 101 actuates an object on the user interface 111that reads, “yes, I would like to take a picture now.”

In block 530, the service application 112 activates the camera component116 on the user computing device 110 and the user 101 captures a facialimage of himself or of herself. In an example, the user computing deviceuser interface 111 may display a live camera feed of the user 101 to aidthe user 101 in aligning the user's 101 face to take the facial image.In an example, the service application 112 may display on the usercomputing device 110 a box or other perimeter on the user interface 111within which the user 101 should align his face to take a picture of arequired size predetermined by the account management system 150. In anexample, the user 101 may actuate an object on the user interface 111 tocapture the image. In this example, in response to the user actuatingthe object on the user interface 111, the camera component 116 receivesa command from the service application 112 to capture an image of theuser 101. In another example, the camera component 116 receives acommand from the service application 112 to capture a plurality ofimages of the user 101 as the user 101 moves the camera around theuser's 101 face. For example, each of the plurality of images of theuser 101 may correspond to a particular pose of the user's 101 face. Anexample facial image may comprise a digital image of the face of a user101. In an example, the account management system 150 may establishguidelines for users 101 in submitting facial images. For example, theservice application 112 may direct the user 101 to remove any hats, headcoverings, glasses, or other objects or accessories that may occluderegions of the user's 101 face so that service application 160 mayreceive a complete depiction of the user's 101 face.

In an example, the user computing device 110 determines if the capturedfacial image is a valid facial image or an invalid facial image. Forexample, a valid facial image complies with guidelines predetermined bythe account management system 150 and an invalid facial image does notcomply with one or more of the guidelines. For example, if the usercomputing device 110 captures a facial image that comprises incorrectdimensions, if part or all of the user's 101 face is occluded, or if theimage is too dark or too bright, the user computing device 110 rejectsthe invalid facial image and displays a request directing the user 101to capture a subsequent facial image. In this example, the user 101captures a subsequent facial image via the user computing device 110,and the user computing device 110 transmits the subsequent facial imageto the account management system 150 via the network 140.

In block 540, the account management system 150 receives the facialimage. In another example, the account management system 150 receives aplurality of facial images of the user 101. For example, the serviceapplication 112 transmits the one or more facial images of the user 101to the account management system 150 via the network 140. In an example,the account management system 150 associates the received one or morefacial images with the user 101 account. For example, the accountmanagement system 150 is able to identify the user 101 account toassociate with the received one or more images because the user 101 iscurrently logged in to the service application 112 on the user computingdevice 110 at the time the one or more facial images are transmitted tothe account management system 150. In certain examples, the accountmanagement system 150 determines if the received facial image is a validfacial image or an invalid facial image. For example, a valid facialimage complies with all guidelines predetermined by the accountmanagement system 150 and an invalid facial image does not comply withone or more of the guidelines. For example, if a user 101 submits afacial image that comprises incorrect dimensions, if part or all of theuser's 101 face is occluded, or if the image is too dark or too bright,the account management system 150 rejects the invalid facial image andtransmits a request to the user computing device 110 directing the user101 to capture a subsequent facial image to transmit to the accountmanagement system 150. In this example, the user computing device 110receives and displays the request, the user 101 captures a subsequentfacial image via the user computing device 110, and the user computingdevice 110 transmits the subsequent facial image to the accountmanagement system 150 via the network 140.

In block 550, the account management system 150 creates a facialtemplate associated with the user 101 account based on the receivedfacial image. In another example, the account management system 150generates a corresponding facial template for each of a plurality ofreceived facial images associated with the user 101 account. In anexample, the facial template is of a predetermined size, for example, a128-byte facial template. In an example, the account management system150 generates a facial template comprising a computer coderepresentation of the digital facial image. For example, the facialtemplate may describe key features of the facial image of the user 101,such as shape, color, line, value, space, form, texture, or other usefulor relevant feature of the image or of particular regions of the image.In an example, the facial template is generated by processing the facialimage through a convolutional neural network. In an example, the accountmanagement system 150 stores the generated facial template associatedwith the user 101 in a data storage unit 155 associated with the accountmanagement system 150. For example, the account management system 150database may comprise a table or other means by which it correlates eachuser 101 account identifier with an associated facial template of theuser 101.

In another example, after the user computing device 110 captures one ormore facial images of the user 101, the user computing device 110generates one or more facial templates corresponding to one or more ofthe one or more captured facial images of the user 101. In this example,the user computing device 110 transmits the one or more generated facialtemplates to the account management system 150 over the network 140.

In block 560, the account management system 150 deletes the receivedfacial image. For example, the account management system 150 only uses afacial template comprising a computer code representation of the facialimage of the user 101. In another example, the account management system150 saves the received facial image for future processing. For example,the account management system 150, at a later time, updates a facialtemplate generation algorithm and generates an updated facial templatecorresponding to the saved facial image.

From block 560, the method 440 proceeds to block 230 in FIG. 2.

Returning to block 230, in FIG. 2, the user 101 enters the serviceprovider system location and signs into the service application 112 onthe user computing device 110. In another example the user 101 signsinto the service application 112 at a time before entering the serviceprovider system location and enters the service provider locationcarrying the user computing device 110 signed into the serviceapplication 112.

In block 240, the user device receives a service beacon device 120identifier. The method for receiving, by a user computing device 110, aservice beacon device 120 identifier broadcast by a service beacondevice 120 is described in more detail hereinafter with reference to themethod 240 described in FIG. 6.

FIG. 6 is a block diagram depicting a method 240 for receiving, by auser computing device 110, a service beacon device 120 identifierbroadcast by a service beacon device 120, in accordance with certainexamples. The method 240 is described with reference to the componentsillustrated in FIG. 1.

In block 610, the user 101 signs in to the service application 112 onthe user computing device 110. In an example, the user 101 may have ausername and password associated with the user 101 account maintained bythe account management system 150. In an example, the user 101 opens theservice application 112 on the user computing device 110 and enters ausername and/or password via the user interface 111 to sign in to theservice application 112. In an example, when the user 101 is signed into the service application 112, the service application is able tocommunicate with the account management system 150 over the network 140.In this example, when the user 101 is not signed in to the serviceapplication 112, the service application does not communicate with theaccount management system 150 even if the a network 140 connection isavailable. In an example, the user 101 may sign out of the serviceapplication 112 at any time by actuating one or more objects on the userinterface 111 of the user computing device 110. In an example, aftersigning in to the service application 112, the user 101 configure one ormore user 101 account settings, add, edit, or delete user 101 paymentaccount information, and/or change user 101 preferences. In certainexamples, a user 101 may be required to make a feature selection toobtain the benefits of the techniques described herein. For example, theuser 101 may have to enable one or more user 101 account settings toenable hands free transactions according to the methods describedherein.

In an example, service application 112 may provide options, data,configurable alerts, and other suitable features to the user 101. Forexample, the service application 112 may comprise a listing of serviceprovider systems and service provider locations that participate inhands free payment transactions according to one or more of the methodsdescribed herein. The listing may be updated periodically from theaccount management system 150. The service application 112 may notifythe user 101 when the user 101 is within a configured vicinity of aparticipating service provider system. The service application 112 mayprovide the user 101 with options to update payment preferences. Theservice application 112 may provide the user 101 with a listing ofrecent transactions. The service application 112 may provide any othersuitable information to the user 101.

In block 620, the user 101 carries the user computing device 110 withina threshold distance of the service beacon device 120 at the serviceprovider system location. In an example, the user 101 enters a locationof the service provider system. The user 101 may enter the serviceprovider location carrying the user computing device 110 in a pocket ora bag, in the hands of the user 101, or in any suitable manner. Thelocation of the service provider system may be a store location, a kiosklocation, or any suitable physical location of a service providersystem. In another example, a service provider POS operator 102 may bemobile and arrive at a location of the user 101. For example, theservice provider system may be a restaurant and the service provider POSdevice operator 102 may be a delivery person possessing a portableservice computing device 130.

In certain examples, the service application 112 may alert the user 101when the user 101 is in the vicinity of a service provider system thataccepts hands-free payments. The alert may be provided via a message onthe user computing device 110, via an email or a text, or in anysuitable manner. In an example, the alert may be based on the locationof the user 101 as determined by a GPS module 119 resident on the usercomputing device 110. For example, the service application 112 accessesthe GPS data from the GPS module 119 and compare the GPS location to alist of locations of service provider systems that accept hands freepayments. For example, the service application 112 comprises a list oraccesses a list maintained by the account management system 150 ofservice provider system locations that accept hands free payments. If amatch results from the comparison, then an alert is generated andprovided to the user 101. The match may result if the user 101 is withina configured distance of a qualified service provider system location.In an example, the alerts may be configured to alert in any suitablemanner. In an example, the alerts may be combined in commercially denseenvironments or the alerts may be presented individually. In anotherexample, the alerts may be configured to only alert the user 101 aconfigured number of times. For example, the alert may be presentedthree times, but upon a fourth instance, the alert is not presented. Thealerts may be presented as a notification with an audible alert, avibration, a popup alert on the user interface 111 of the user computingdevice 110, or other suitable alert.

In block 630, the user computing device 110 receives a service beacondevice 120 identifier broadcast by the service beacon device 120. Theuser computing device 110 recognizes a service beacon device 120 viawireless communication at the location of the service provider system.The user computing device 110 may be configured to search for beacons orother wireless signals. In an example, the user computing device 110 andthe service beacon device 120 establish a Wi-Fi wireless network 140connection. In other examples, the user computing device 110 and theservice beacon device 120 establish a Bluetooth, BLE, NFC, or otherappropriate network 140 connection Upon entering the range of the signalof the service beacon device 120, the user computing device 110 receivesthe service beacon device 120 identifier.

In block 640, the user computing device 110 transmits the receivedservice beacon device 120 identifier and a user 101 account identifierto the account management system 150. In an example, the user computingdevice 110 transmits the data received in the service beacon device 120identifier along with a user 101 account identifier to the accountmanagement system 150 over the network 140.

In block 650, the account management system 150 receives the servicebeacon device 120 identifier and the user 101 account identifier. Forexample, the account management system 150 receives the service beacondevice 120 identifier and the user 101 account identifier over thenetwork 140. The user computing device 110 may compare the data from theservice beacon device 120 identifier to a database of service beacondevice 120 identifier data and service camera device identifier data todetermine an identity of the service provider system and service cameradevice 135 associated with the service beacon device 120 identifierand/or to verify the authenticity of the beacon.

From block 750, the method 240 proceeds to block 250 in FIG. 2.

Returning to FIG. 2, in block 250, the service computing device 130receives a facial template for each user 101 in range of the servicebeacon device 120. The method for receiving, by a service computingdevice 130, a facial template for each user 101 in range of the servicebeacon device 120 is described in more detail hereinafter with referenceto the method 250 described in FIG. 7.

FIG. 7 is a block diagram depicting a method 250 for receiving, by aservice computing device 130, a facial template for each user 101 inrange of the service beacon device 120, in accordance with certainexamples. The method 250 is described with reference to the componentsillustrated in FIG. 1.

In block 710, the account management system 150 extracts a facialtemplate associated with the user 101 account identifier. In an example,the account management system 150 accesses a database comprising storedfacial templates of a plurality of users 101 with corresponding user 101account identifiers for each user 101. For example, this database isstored in the data storage unit 155.

In block 720, the account management system 150 identifies a servicecomputing device 130 associated with the service beacon device 120identifier. In an example, the account management system 150 recognizesthat the service beacon device 120 identifier is associated with theaccount management system 150 and a particular service computing device130 at the service provider system location. In an example, the accountmanagement system 150 recognizes that the service beacon device 120identifier is associated with a plurality of service camera devices 140installed at a particular service provider location.

In block 730, the account management system 150 transmits the facialtemplate of the identified user 101 to the service computing device 130associated with the service beacon device 120 identifier. In anotherexample, the account management system 150 transmits the facial templateof the identified user 101 to a plurality of service computing devices130 associated with the service beacon device 120 identifier. In certainexamples, the account management system 150 receives, in real time, aplurality of transmissions from user computing devices 101 correspondingto a plurality of users 101 present at the service provider systemlocation, each transmission comprising a user 101 account identifier anda retransmitted service beacon device 120 identifier. In these examples,the account management system 150 retrieves, in response to receivingeach such transmission, a facial template associated with the receiveduser 101 account identifier and transmits a facial template to one ormore service camera devices 140 at the service provider locationassociated with the service beacon device 120 identifier.

In block 740, the service computing device 130 receives the facialtemplate of the user 101. In another example, a plurality of servicecomputing devices 130 receive the facial template of the user 101. Inyet another example, the service computing device 130 and/or theplurality of service computing devices 130 receive one or moreadditional facial templates from the account management system 150corresponding to one or more users 101 other than the instant user 101having user computing devices 110 in network 140 connection to a servicebeacon device according to the method previously described herein. Forexample, the one or more additional facial templates are received inreal time from the account management system 150 as additional users 101other than the instant user 101 receive the service beacon device 120identifier over a wireless communication network 140 or otherwiseestablish a network 140 connection between their user computing devices110 and one or more service beacon devices 120. For example, the one ormore service camera devices 140 may receive one or more additionalfacial templates corresponding to one or more additional users 101 at atime before, at the same time, or after the time at which the servicecomputing device 130 receives the facial template of the instant user101.

In block 750, the service camera computing device 130 adds the facialtemplate of the user 101 to a current customer log. In an example, thecurrent customer log is accessible by the service computing device 130and by the account management system 150. In another example, thecurrent customer log is additionally accessible by the service computingdevice 130 and/or an applicable computing device of the service providersystem. In an example, the account management system 150 comprises andmaintains the current customer log, which the service computing device130 may access via the network 140. In another example, the servicecomputing device 130 comprises and maintains the current customer log,which is accessible to the account management system 150 bycommunicating with the user computing device 110 via the network 140.

In block 760, the service computing device 130 periodically updates thecurrent customer log. The method for receiving, by a service cameradevice 135, notification from a account management system 150 as users101 enter or leave a network range of a service beacon device 120 isdescribed in more detail hereinafter with reference to the method 760described in FIG. 8.

FIG. 8 is a block diagram depicting a method 760 for receiving, by aservice camera computing device 130, notification from a accountmanagement system ISO as users 101 enter or leave a network range of aservice beacon device 120, in accordance with certain examples. Themethod 760 is described with reference to the components illustrated inFIG. 1.

In block 810, the service computing device 130 is notified by theaccount management system ISO as users 101 signed into a service accountenter or leave a network range of the service beacon device 120. Forexample, as previously discussed, when a user 101 carrying a usercomputing device 110 enters a threshold distance from a service beacondevice 120, the service beacon device 120 or the user computing device110 of the user 101 are able to detect the other device and establish awireless network 140 connection between the two devices at the serviceprovider system location. In this example, the service beacon device 120transmits the service beacon device 120 identifier corresponding to theservice beacon device 120 over the wireless network 140 to the usercomputing device 110. For example, the service beacon device 120transmits the service beacon device 120 identifier to the user computingdevice 110 via a Wi-Fi, Bluetooth, BLE, or NFC wireless communicationnetwork 140. In this example, the user computing device 110 retransmitsthe received service beacon device 120 identifier to the accountmanagement system 150 along with a user 101 account identifieridentifying the user 101.

In block 820, the service computing device 130 determines whether a newuser 101 is in range of the service beacon device 120. For example, ifthe account management system 150 receives a new user 101 accountidentifier in addition to the same service beacon device 120 identifier,the account management system 150 may determine that a new user 101 isin range of the service beacon device 120. In this example, the accountmanagement system 150 may infer that the new user 101 has entered theservice provider location based on receipt of the new user 101 accountidentifier. In another example, if the account management system 150does not receive any new user 101 account identifiers along with thesame service beacon device 120 identifier within a threshold length oftime, the account management system 150 may determine that no new users101 have entered the network 140 range of the service beacon device 120.

If a new user 101 is in range of the service beacon device 120, themethod 860 proceeds to block 830 in FIG. 8. For example, the accountmanagement system 150 receives a new user 101 account identifier inaddition to the same service beacon device 120 identifier. In thisexample, the account management system 150 infers that the new user 101has entered the service provider location based on receipt of the newuser 101 account identifier and the same service beacon device 120identifier as previously received from the first user 101.

Returning to FIG. 8, in block 810, the account management system 150extracts a facial template associated with the new user 101 accountidentifier. In an example, the account management system 150 transmitsthe facial template to the appropriate one or more service computingdevices 130 and the one or more service computing devices 130 add thenew user's 101 facial template to the current customer log according tothe example method previously described in method 250 in FIG. 7.

Returning to FIG. 8, in block 820, if there is a not a new user 101 inrange of the service beacon device 120, the method 760 proceeds to block830. For example, the account management system 150 does not receive anynew user 101 account identifiers along with the same service beacondevice 120 identifier within a threshold length of time and determinesthat no new users 101 have entered the network 140 range of the servicebeacon device 120.

In block 830, the service camera device 135 determines whether a currentuser 101 has moved out of range of the service beacon device 120. In anexample, the user computing device 110 continues to receive from theservice beacon device 120 identifier from the service beacon device 120and retransmit the service beacon device 120 identifier along with theuser 101 account identifier to the account management system 150. Inthis example, the user computing device 110 may periodically transmitinformation comprising the service beacon device 120 identifier and user101 account identifier to the account management system 150 as long asthe user computing device 110 continues to detect the service beacondevice 120 and receive the service beacon device 120 identifier viaperiodic scans. For example, the user computing device scans for theservice beacon device 120 every five seconds. In another example, theuser computing device 110 may periodically transmit informationcomprising the service beacon device 120 identifier and user 101 accountidentifier to the account management system 150 as long as the usercomputing device 110 maintains a wireless network 140 connection withthe service beacon device 120. For example, the user computing device110 may transmit this information to the account management system 150at every five seconds. In this example, if the account management system150 ceases to receive the information from the user computing device fora predefined number of intervals, the account management system 150 maydetermine that the corresponding user 101 has moved out of range of theservice beacon device. In this example, if the account management system150 continues to receive the information transmitted by the usercomputing device 110 at the expected intervals, the account managementsystem 150 determines that the user 101 is still in network 140 range ofthe service beacon device 120.

If no current user 101 has moved out of range of the service beacondevice 120, the method 860 proceeds to block 260 in FIG. 2. For example,the account management system 150 continues to receive the servicebeacon device 120 identifier and user 101 account identifier transmittedby the user computing device 110 at the expected intervals anddetermines that the user 101 is still in network 140 range of theservice beacon device 120.

Returning to block 260, in FIG. 2, the service computing device 130identifies the user 101 via facial recognition.

Returning to FIG. 8, in block 830, if a current user 101 has moved outof range of the service beacon device 120, the method 760 proceeds toblock 840.

In block 840, the service computing device 130 receives a notificationfrom the account management system 150 that a current user 101 is out ofnetwork range of the service beacon device 120. In another example, theservice computing device 130 receives a notification from the accountmanagement system 150 that the user computing device 110 associated withthe current user 101 has stopped sending notifications to the accountmanagement system ISO comprising the service beacon device 120identifier. For example, the service computing device 130 receives theuser 101 account identifier associated with the current user 101associated with a user computing device 110 that is either out ofnetwork range or has stopped transmitting notifications comprising theservice beacon device 120 identifier to the account management system150, accesses the current customer log, and finds an entry correspondingto the current user 101. For example, the current customer log ismaintained by the account management system 150 and the servicecomputing device 130 accesses the current customer log over the network140 by communicating with the account management system 150. In anotherexample, the account management system 150 does not transmit anotification to the service computing device 130 that the current user101 is out of network range. In this example, the account managementsystem 150 accesses the current customer log and deletes the facialtemplate of the current user 101 from the current customer log.

In block 850, the service computing device 130 deletes the facialtemplate of the current user 101 from the current customer log. Forexample, the current customer log comprises a table and the servicecomputing device 130 deletes or requests the deletion of an entry or rowcorresponding to data associated with the current user 101 for which theservice camera device 135 received the notification. In another example,the account management system 150 accesses the current customer log anddeletes the facial template of the current user 101 from the currentcustomer log.

From block 850, the method 760 proceeds to block 260, in FIG. 2.

Returning to FIG. 2, in block 260, the service computing device 130identities the user 101 via facial recognition. The method foridentifying, via a service computing device 130, a user 101 via facialrecognition is described in more detail hereinafter with reference tothe method 260 described in FIG. 9.

FIG. 9 is a block diagram depicting a method 260 for identifying, via aservice computing device 130, a user 101 via facial recognition, inaccordance with certain examples. The method 260 is described withreference to the components illustrated in FIG. 1.

In block 910, the service computing device 130 captures video feed ofthe user 101. In an example, in response to receiving a request toidentify the user 101, the service computing device 130 activates thecamera component 136 of a service camera device 135 to begin to capturea video of the surroundings of the service camera device 135. In anexample, the service camera device 135 captures a video feed. In anotherexample, the service camera device 135 continuously captures, but doesnot record, a video feed of its surroundings. In this example, when theservice camera device 135 receives a request to identify the user 101from the account management system 150, the service camera device 135beings to record the video feed for a threshold amount of time. In anexample, the user 101 may be moving during the period in which theservice camera device 135 records the video feed. In an example, aprocessor 137 of the service camera device 135 extracts a facial imageby determining a particular frame of the video feed and area of theinstance of the video feed corresponding to the face of the user 101.

In block 920, the service computing device 130 extracts a facial imageof the user 101 from the captured video feed. In an example, the servicecomputing device 130 determines a frame of the captured video to providean image of the user's 101 face and extracts the frame of the capturedvideo comprising the facial image of the user 101.

In certain other examples, the service computing device 130 determines aframe of the captured video to provide an image of the faces of aplurality of users 101. For example, the frame comprises an image of theface of a first user 101, a second user 101, and a third user 101 atdifferent locations in the image. In this example, one service cameradevice 135 may capture video of an environment corresponding to an areain the proximity of multiple service computing devices 130. In thisexample, the service computing device 130 may determine to whichparticular service computing device 130 each of the plurality of facesof the corresponding plurality of users 101 in the extracted image.

In an example, in which a service camera device 135 is mounted at anangle parallel to a plurality of adjacent service computing devices 130,the field of view of the service camera device 135 may be divided intobounding boxes such that each bounding box is assigned to a particularservice computing device 130. In this example, since one or more users101 may be situated between any two adjacent service computing devices130, the bounding boxes may overlap. In this example, each bounding boxis assigned to at most two adjacent service computing devices 130. Inthis example, in which the service camera device 135 is mounted parallelto a plurality of adjacent service computing devices 130, the size of adetected facial image of a user 101 is directly proportional to thedistance the user 101 is from a service computing device 130 detected inthe image. In this example, the service computing device 130 assignseach detected facial image to one or two particular service computingdevices 130 based on the position of the facial image relative to thebounding boxes.

In another example, the service camera device 135 is mounted at an anglethat is not parallel to a plurality of adjacent service computingdevices 130. In this example, the service camera device 135 may be ableto monitor an environment comprising a greater number of adjacentservice computing device 130 than the example in which the servicecamera device 135 is mounted parallel to adjacent service computingdevices 130 in this example, however, the size of a detected facialimage is not proportional to the distance between a service computingdevice 130 detected in the image and the detected facial image. In thisexample, the service computing device 130 assigns each detected facialimage to one or two particular service computing devices 130 based onthe position of the facial image relative to the bounding boxes.

In block 930, the service computing device 130 generates a facialtemplate from the captured facial image. In an example, the facialtemplate is of a predetermined size, for example, a 128-byte facialtemplate. In an example, the account management system 150 generates afacial template comprising a computer code representation of the digitalfacial image. For example, the facial template may describe key featuresof the facial image of the user 101, such as shape, color, line, value,space, form, texture, or other useful or relevant feature of the imageor of particular regions of the image. In another example, the facialtemplate is generated by processing the facial image through aconvolutional neural network In an example, the service computing device130 stores the generated facial template in a data storage unit 139associated with the service camera device 135 and the service computingdevice 130. For example, the service computing device 130 database maycomprise a log of facial templates of current customers wherein theservice computing device 130 stores the generated facial template.

In certain other examples, the service camera device 135 continuouslycaptures a video feed of its surroundings as users 101 enter and leavethe vicinity of one or more service computing devices 130 over thecourse of a certain time period. In this example, the service cameradevice 135 processor 137 is able to continuously monitor the incomingvideo feed to detect faces from extracted frames of the video feed. Inthis example, the service camera device 135, each time the processor 137detects the presence of one or more faces in the video feed, the servicecomputing device 130 extracts a frame of the video feed comprising oneor more facial images of one or more corresponding detected faces andcreates facial templates based on the extracted one or more facialimages. In this example, the service computing device 130 stores facialtemplates in the log of facial templates of current customers as theyare generated. In this example, as the service computing device 130generates a subsequent facial templates, the service computing device130 determines whether the generated subsequent facial template issimilar to within a threshold compared to any of the facial templatesalready stored in the log of facial templates of current customers. Ifthe generated subsequent facial template is similar to within athreshold to any of the facial templates already stored in the log, theservice computing device 130, after associating the facial template toone or two particular service computing devices 130 based on theposition of the associated facial images in the extracted frame of thecaptured video, adds the facial template to the log of facial templatesof current customers. If the generated subsequent facial template is notsimilar to within a threshold to any facial templates already stored inthe log of facial templates of current customers, the service computingdevice 130 deletes or otherwise ignores and/or does nothing with thegenerated facial template. In this example, if the service computingdevice 130 determines that certain facial image is no longer in thefield of the video feed, the corresponding facial template is deletedfrom the log of facial templates of current customers.

In block 940, the service computing device 130 deletes the capturedvideo and the extracted facial image. For example, the service computingdevice 130 does not store captured images or video. In this example,facial templates generated by the service computing device 130 comprisecomputer code representations of facial images of users 101. In thisexample, after generating a facial template or after a threshold timehas passed after capturing video or images or extracting an image from avideo, the service computing device 130 deletes any captured orextracted video or images.

In block 950, the service computing device 130 retrieves facialtemplates from the current customer log. For example, the currentcustomer log comprises facial templates received from the accountmanagement system 150 corresponding to all current users 101 whoseassociated user computing devices 110 are located within a networkdistance of a service beacon device 120.

In block 960, the service computing device 130 compares the generatedfacial template from captured facial image to facial templates from thecurrent customer log.

In block 970, the service computing device 130 determines whether thereis a match between the generated facial template and one of the facialtemplates from the current customer log. In this example, when theservice computing device 130 determines a match between a first facialtemplate in the log of facial templates of current customers and asecond facial template the log of current customers, the servicecomputing device 130 determines with which service provider one or twoservice computing devices 130 the second facial template is associated.In this example, the service computing device 130 updates the entry ofthe second facial template in the log of current customers with anannotation assigning the second facial template to a particular one ortwo service computing devices 130. For example, as previously discussed,the facial image upon which the facial template was located within abounding box of an area of the video or image corresponding to one ortwo particular service computing devices 130. In this example, when theservice computing device 130 generates a facial template for the facialimage, the service computing device 130 associates the generated facialtemplate with the particular one or two service computing devices 130.For example, if the facial image was in a bounding box in front of afirst service computing device 130, the resulting facial template may beassociated with the first service computing device 130. In anotherexample, if the facial image was in a bounding box between the firstservice computing device 130 and a second, adjacent service computingdevice 130, the resulting facial template may be associated with boththe first service computing device 130 and the second service computingdevice 130.

If a facial template from the current customer log matches the generatedfacial template, the method 260 proceeds to block 980. In anotherexample, if a facial template from the current customer log matches afacial template from the log of facial templates of current customers,the method 260 proceeds to block 980.

In block 980, the service computing device 130 transmits the facialtemplate from the current customer log corresponding to the identifieduser to the account management system 150. For example, the transmittedfacial template comprises an annotation assigning the facial template toa particular one or two service computing devices 130.

From block 980, the method 260 proceeds to block 270 in FIG. 2. In thisexample, the account management system 150 identifies the user 101requesting a service request and a particular service computing device130 to which the user 101 is assigned and may proceed with processingthe transaction.

Returning to block 970, if none of the facial templates from the currentcustomer log matches the generated facial template, the method 260proceeds to block 270 in FIG. 2. In another example, if none of thefacial templates from the current customer log matches any of the facialtemplates from the log of facial templates of current customers, themethod 270 proceeds to block 270 in FIG. 2.

Returning to FIG. 2, in block 270, the service computing device 130identities the user 101 via movement recognition. The method foridentifying, via a service computing device 130, a user 101 via movementrecognition is described in more detail hereinafter with reference tothe method 270 described in FIG. 10. In an example, the method 270 foridentifying, by a service computing device 130, users 101 in a videofeed based on movement data received from user computing devices 110 maybe performed instead of or in addition to the method 260 for identifyingusers 101 in a video feed based on facial recognition. In otherexamples, the account management system 150 compares the video feed withthe movement data to identify users 101 in the video feed.

FIG. 10 is a block diagram depicting a method 270 for identifying, via aservice computing device 130, a user 101 via movement recognition, inaccordance with certain examples. The method 270 is described withreference to the components illustrated in FIG. 1.

In block 1010, the user computing device 110 logs movement data andlocation data. Example movement data comprises accelerometer 118 dataand/or gyroscopic data of the user computing device 110, a usercomputing device 110 identifier, and timestamp data. For example,movement data comprises a velocity of the user computing device 110, anacceleration of the user computing device 110, an orientation of theuser computing device 110, a rotation rate of the user computing device110, or other data that describes or indicates an orientation, position,or a movement of the user computing device 110. For example, timestampdata comprises a current time comprising a year, month, date, hour ofthe day, minute of the hour, second of the minute, millisecond of thesecond, and/or other appropriate indication or indications of the timeat which the timestamp is generated. Example location data comprises alongitude and latitude logged by the GPS module 119 of the usercomputing device 110.

In block 1020, the user computing device 110 transmits the loggedmovement and the location data to the service computing device 130 whenthe user 101 is signed into the service application 112. In an example,each user computing device 110 continuously or periodically transmitslogged movement data to the service computing device 130 via a wirelesscommunication channel if the user is signed in to the serviceapplication 112. In an example, if the user 101 is not signed in to theservice application 112, the user computing device 110 does not transmitlogged movement data to the service computing device 130 via thewireless communication channel. In other examples, the user computingdevice 10 continuously or periodically transmits logged movement data tothe account management system 150 via the network 140 when the user 101is signed in to the service application 112 on the user computing device110. For example, in addition to transmitting a beacon device identifierto the account management system 150, each user computing device 110 atthe service system location transmits movement data logged by the usercomputing device 110 to the account management system 150.

In block 1030, the service computing device 130 receives the movementdata and the location data from the user computing device 110. In anexample, the service computing device 130 receives the movement datafrom user computing devices 110 at the service provider location via thewireless communication channel. In another example, the servicecomputing device 130 receives the movement data and the location datafrom the account management system 150, which received the movement dataand the location data from each user computing device 110.

In block 1040, the service computing device 130 compares the receivedmovement data and the received location data to the current video feedof the service computing device 130. For example, the service computingdevice 130 compares a received video feed with received movement data ofuser computing devices 110 to identify users in the video feed. In anexample, the service computing device 130 receives a continuous videofeed from the service camera device 135 comprising video captured of anarea of the service provider system location. Example video feed datacomprises timestamp data. In an example, the service computing device130 defines a location of the area captured by the video feed associatedwith a service request area and determines an identity of the usercomputing device 110 within the service request area based on a receiveduser computing device 110 identifier. In an example, the video feedcomprises multiple service request areas, each service request areaassociated with a particular service computing device 130. For example,the account management system 150 receives the movement data from eachuser computing device 110 at the service provider system location viathe network 140. In this example, the account management system 150receives a video feed from the service computing device 130 and comparesthe received video feed with the movement data of user computing devices110 to identify users 101 in the video feed. In this example, theaccount management system transmits identities of the users identifiedin the video feed to the service computing device 130. In anotherexample, the account management system 150 transmits the receivedmovement data to the service computing device 130 it the serviceprovider location In this other example, the service computing device130 compares the received video feed with movement data of usercomputing devices 110 received from the account management system 150 toidentify users 101 in the video feed. In an example, the servicecomputing device 130 defines a location of the area captured by thevideo feed associated with a service request area and determines anidentity of the user 101 within the service request area. In an example,the video feed comprises multiple service request areas, each servicerequest area associated with a particular service computing device 130.In this example, the service computing device 130 or account managementsystem 150 determines an identity of a particular user 101 within eachservice request area associated with each particular correspondingservice computing device 130.

In an example, the service computing device 130 or the accountmanagement system 150 may be trained to identify users 101 in a videofeed based on movement data received from user computing devices 110.For example, a service application 130 on each user computing device 110at the service provider system location logs data from accelerometers118 and/or gyroscopes and sends that data via a wireless communicationchannel to a processing unit of the service computing device 130. Theprocessor simultaneously records a video feed from a service cameradevice 135 communicatively coupled to the service computing device 130.In other examples, the service computing device 130 comprises theservice camera device 135. In an example, the processing unit 137 uses amachine learning model to identify similarities between gyroscopic andaccelerometer 118 data and visual data from the video feed received fromthe service camera device 135. In an example, the service computingdevice 110 uses previous identifications of user computing device 110identifiers corresponding to objects video feeds based on correlation ofmovement data to changes in position of an object in the video feedidentified as a potential user 101 to train the machine learning modelto identify similarities between gyroscopic and accelerometer 118 dataand visual data from the video feed In an example, the machine learningmodel identifies similarities between gyroscopic and accelerometer 118data comprising and visual data from the video feed that corresponds tothe same time period as the received gyroscopic and accelerometer 118data. In this example, the time period may be determined based on timestamp data associated with the received video feed data and the receivedgyroscopic and accelerometer 118 data for each user computing device110. For example, if a user 101 were to pull his or her user computingdevice 110 out of his or her pocket at a time period A, the servicecomputing device 130 would identify a signature both in the video feedas well as a corresponding signature from the received movement datacomprising the gyroscopic and the accelerometer 118 data during the sametime period A. The processing unit would associate the changes in themovement data with changes in the video feed data during the time periodand would associate a user computing device 110 identifier with aservice request area in the video feed for the time period. In example,the time period comprises a time period that terminates at the currenttime. In an example, the service request area corresponds to a locationin front of the service computing device 130. The processing unit wouldidentify the user 101 based on the user computing device 110 identifieror receive, from the account management system 150, an identity of theuser 101 in response to transmitting a request for the identity of theuser 101 to the account management system along with the user computingdevice 110 identifier in the service request area.

For example, the machine learning system is structured as follows: theservice computing device 130 or account management system 150 entersreceived gyroscopic data and accelerometer 118 data for each usercomputing device 110 at the location into a Long Short Term Memory(“LSTM”) Model and a Deep Neural Network (“DNN”) Classifier to comparedata from the video feed to the gyroscopic data and accelerometer 118data and detect users 101 within the video feed. In an example, theservice computing device 130 or the account management system 150 usesthe DNN classifier to detect potential users 101 within the video feedbased on object recognition. For example, the DNN classifier executes aperson recognition algorithm on the service computing device for eachframe of the video feed to determine where each user 101 is located inthe fame. In an example, the service computing device 130 or the accountmanagement system 150 uses the DNN Classifier to determine whether aregion of the video feed that corresponds to a potential user 101 isassociated with a particular user computing device 110 represented bythe received user computing device 110 movement data. In an example, theLSTM Model operates on the service application 112 on each usercomputing device 110 at the service provider system location to generatea serializable encoding of gyroscopic signatures and accelerometersignatures. The service computing device 130 or the account managementsystem 150 uses the DNN Classifier to compare a serializable encoding ofgyroscopic signatures and accelerometer 118 signatures with the visualdata for each detected person in the video feed to determine asimilarity score. In an example, if the similarity score for aparticular user computing device 110 identifier is greater than apredetermined threshold, the service computing device 130 or the accountmanagement system 150 uses the DNN classifier to determine that the usercomputing device 110 identifier corresponds to the detected person inthe video feed.

In other examples, instead of determining the user computing device 110identifier that corresponds to the detected person in the video feedbased on a similarity score being greater than a predeterminedthreshold, the DNN classifier uses a tiered threshold matching system.In an example tiered threshold matching system, the DNN determines astrong match and a weak match for each detected person in the videofeed. The example strong match comprises a higher threshold similarityscore than the example weak match so as to maximize precision. Forexample, a similarity score is determined based on comparing a signaturefrom the video feed as well as a corresponding signature from thereceived movement data comprising the gyroscopic and the accelerometer118 data during a time period. The example weak match comprises athreshold similarity score low enough so that an actual present matchwould be a weak match. In an example, the DNN classifier generatesstrong and weak matches for all detected objects detected to be personsin the video feed and tracks a unique strong match for a detectedobject. If a strong match is not present, the DNN classifier tracks aunique weak match for the detected object for all frames of the videofeed. Once all detected objects resembling potential persons aretracked, the DNN classifier determines if multiple user computingdevices 110 match a single tracked object in any frame of the videofeed. Any frames of the video feed for the single tracked object forwhich the tracked object is assigned to more than one user computingdevice 110 are excluded from consideration when determining movement ofthe object in the video feed.

In block 1050, the service computing device 130 determines if a user 101can be identified in the video feed based on comparing the receivedmovement data and the received location data to the current video feedof the service computing device 130. In an example, determining if auser 101 can be identified in the video feed comprises determiningwhether the user 101 can be identified in the video feed within aservice area of the video feed.

If the service computing device 130 does not identify a user 101 in thevideo feed, the method 270 returns to block 1030. In block 1030, theservice computing device 130 receives subsequent movement data andsubsequent location data from the user computing device 110.

Returning to block 1050, if the service computing device 130 identifiesa user 101 in the video feed, the method 270 proceeds to block 1060. Inan example, the service computing device 130 uses the DNN Classifier todetermine whether a region of the video feed that corresponds to apotential user 101 is associated with the user computing device 110represented by the received user computing device 110 movement data. Inan example, the LSTM Model operates on the service application 112 onthe user computing device 110 at the service provider system location togenerate movement data comprising a serializable encoding of gyroscopicsignatures and accelerometer signatures. The service computing device130 uses the DNN Classifier to compare a serializable encoding ofgyroscopic signatures and accelerometer 18 signatures with the visualdata for the object in the video feed comprising the potential user 101to determine a similarity score In an example, if the similarity scorefor a particular user computing device 110 identifier is greater than apredetermined threshold, the service computing device 130 uses the DNNclassifier to determine that the user computing device 110 identifiercorresponds to the detected user 101 in the video feed.

In block 1060, the service computing device 130 designates a user's 101specific location at the service provider system location based on thecorrespondence between the video feed and the movement data and thelocation data. In an example, the user's 101 specific location at theservice provider location is designated as being in a service area ifthe correspondence of gyroscopic and/or accelerometer 118 data of theuser computing device 110 with the video feed indicates that the user101 is in a service area of the service provider system location. Forexample, the service area comprises an area within a threshold proximityto the service computing device 130 within which the user 101 mayinitiate a service request.

From block 1060, the method 270 proceeds to block 280 in FIG. 2.

Returning to FIG. 2, in block 280, the user 101 initiates a servicerequest at the service computing device 130. For example, a servicerequest comprises a payment transaction. The method for initiating, by auser 101, a service request at a service computing device 130 isdescribed in more detail hereinafter with reference to the method 280described in FIG. 11. In the examples described herein, the user 101initiates a service request comprising a “hands free transaction” at theservice computing device 130. An example hands free transaction does notrequire any interaction with the user computing device 110 on the partof the user 101. In another example, a hands free transaction requiresonly minimal interaction with the user computing device 110 by the user101.

FIG. 11 is a block diagram depicting a method 280 for initiating, by auser 101, a transaction at a service computing device 130, in accordancewith certain examples. The method 280 is described with reference to thecomponents illustrated in FIG. 1.

In block 1110, the user 101 approaches the service computing device 130.In an example, at a time prior to approaching the service computingdevice 130, the user 101 browses the service provider system locationand selects one or more items to purchase. In this example, the user 101may collect the one or more items and carry, or otherwise transport viaphysical basket or shopping cart, the one or more items to the servicecomputing device 130.

In block 1120, the service computing device 130 operator 102 totals theitems of the user 101 for purchase. In an example, the service computingdevice operator 102 scans barcodes attached to the one or more items orotherwise enters descriptions and prices associated with the one or moreitems into the service computing device 130. In an example, afterscanning or manually entering the items into the service computingdevice 130, the service computing device operator 102 actuates an objecton the user interface 131 of the service computing device 130 to orderthe service computing device 130 to total the items. In an example, theservice computing device 130 displays, via the user interface 131, thetotal to the user 101.

In block 1130, the service computing device 130 operator asks the user101 to select a payment option. In an example, the service computingdevice 130 displays one or more payment options that the user 101 mayselect to use in a transaction. Example payment options may comprisepayment via a service application 112 associated with the accountmanagement system 150, payment by cash, payment by check, payment bycredit card, payment by debit card, and/or any other means of paymentthat the service provider system can or is willing to accept for paymentfrom the user 101. In an example, the one or more payment options aredisplayed as objects on the user interface 131 and are selectable by theservice system operator 102 in response to the user 101 directing theservice system operator 102 to make a selection.

In another example, the account management system 150 or service cameradevice 135 transmits a notification to the service computing device 130that the user 101 is able to initiate a transaction via the serviceapplication 112. In this example, the service computing device 130notifies the service system operator 102 that the user 101 is able toparticipate in a service request using the service application 112. Inthis example, the service system operator 102 asks the user 101 toconfirm whether the user 101 wants to initiate a transaction using theservice application 112.

In block 1040, the user 101 directs the service system operator 102 toinitiate a transaction via the service application 112. In an example,in response to receiving a verbal request from the user 101 to selectthe service application 112 as a payment option, the service systemoperator 102 actuates an object on the user interface 131 of the servicecomputing device 130 corresponding to the service application 112payment option.

In block 1150, the service system operator 102 selects an option on theservice computing device 130 to initiate a transaction using the serviceapplication 112. In an example, the service computing device 130displays a confirmation screen after the service system operator 102selects an option to initiate a transaction using the serviceapplication 112. An example confirmation screen may display informationsummarizing the potential transaction and comprising one or more of atransaction total, a description of the one or more items beingpurchased by the user 101, and a indication that the user 101 selectedthe service application 112 as the method of payment for thetransaction. An example confirmation screen may further display optionsto confirm the transaction or cancel the transaction. In an example, theuser 101 reviews the confirmation screen, determines that theinformation displayed on the confirmation screen is correct, determinesto continue with the transaction, and directs the service systemoperator 102 to select the option to confirm the transaction via theuser interface 131.

In another example, the user 101 decides to abandon the transactionbecause the information is incorrect, or because the user 101 changedhis mind and decided not to purchase the items. In yet another example,the confirmation screen further comprises an option to edit thetransaction details. In this example, the service system operator 102,upon direction of the user 101, may select the option to edit thetransaction details and may then edit, add, or delete one or more of theitems in the transaction or edit payment details or payment methods forthe transaction.

In block 1160, the service computing device 130 identifies the user 101from the current customer log. For example, at this point, the servicecomputing device 130 is aware that one or more users 101 are physicallypresent in the service provider system location based on the currentcustomer log maintained by the account management system 150 andaccessible and editable by the service computing device 130.Additionally, the service computing device 130 is aware that a user 101has initiated a transaction with the service application 112 paymentmethod based on the request to process the transaction received from theservice computing device 130. However, the service computing device 130may not be aware of exactly which user 101 requested to process thehands free transaction involving the service application 112. In anexample, the service computing device 130 continuously monitors a videofeed received from the service camera device 135 and determines, inresponse to initiating the service request, that the user 101 is withinthe service area of the video feed based on comparing the movement dataand the accelerometer 118 data to the video feed.

In block 1170, the service computing device 130 transmits a usercomputing device 110 identifier and service request details to anaccount management system 150. For example, in response to receiving aninitiation of the service request and in response to identifying theuser computing device 110 in the video feed as being located within theservice area of the video feed based on correlating movement data andaccelerometer 118 data received from the user computing device 110 todata from the video feed, the service computing device 130 transmits auser computing device 110 identifier to the account management system150 corresponding to the user computing device 110 detected as beingwithin the service area of the video feed. For example, the servicecomputing device 130 transmits a user computing device 110 identifierand service request details to an account management system 150 via thenetwork 140.

In block 1180, the account management system 150 receives the usercomputing device 110 identifier and the service request details. Forexample, the account management system 150 receives the user computingdevice 110 identifier and the service request details via the network110.

In block 1190, the account management system 150 identifies a user 101account identifier associated with the user computing device 110identifier received from the service computing device 130. For example,the account management system 150 manages the user 101 accountassociated the user computing device 110.

From block 1190, the method 280 proceeds to block 290 in FIG. 2.

Returning to FIG. 2, in block 290, an account management system 150processes a service request. The method for processing, by an accountmanagement system 150, a service request is described in more detailhereinafter with reference to the method 290 described in FIG. 12.

FIG. 12 is a block diagram depicting a method 290 for processing, via anaccount management system 150, a service request, in accordance withcertain examples. The method 290 is described with reference to thecomponents illustrated in FIG. 1.

In block 1210, the account management system 150 transmits accountinformation of the identified user 101 to the service computing device130. For example, the user 101 account with the account managementsystem 150 comprises payment account information associated with one ormore payment accounts of the user 101. An example payment account of theuser 101 comprises a credit account, bank account, stored value account,gift card, service provider system charge account, or other relevantuser 101 account. In an example, the service computing device 130 doesnot receive complete payment information for one or more of the user 101payment accounts. For example, the payment information may be obfuscatedor truncated. For example, the service computing device 130 may onlyreceive abbreviated payment information sufficient for a user 101 toidentify a payment account via the user interface 131.

In block 1220, the service computing device 130 displays accounts of theidentified user 101. In an example, the service computing device 130does not display complete account information of the user 101 on theuser interface 131. For example, for a credit card account, the servicecomputing device user interface 131 may display only the name associatedwith the credit card, last four digits of the credit card number, andthe expiration date.

In block 1230, the service computing device operator 102 selects a user101 account for transaction and confirms the transaction with permissionof the user 101. For example, the user 101 views the account optionsdirectly displayed on the user interface 131 or listens to the servicecomputing device operator 102 read the user 101 payment account optionsto the user 101. In an example, the user 101 selects a payment accountoption to use in the transaction. For example, the user 101 decides touse a credit card account. In an example, the user 101 communicates aselection of a payment account for use in the transaction to the servicecomputing device operator 102. In an example, in response to receivingthe user's 101 indication of the payment account, the service computingdevice operator 102 selects the payment account indicated by the user101 on the service computing device user interface 131.

In block 1240, the service computing device 130 transmits transactiondetails to the account management system 150. For example, the servicecomputing device 130 transmits transaction details comprising atransaction total, an account identifier of the service provider system,and the selection of the user 101 account payment option. In an example,the account management system 150 receives the transaction details overthe network 140. In an example, the account management system 150extracts payment account information from the user 101 accountcorresponding to the received selection of the user 101 payment account.

In block 1250, the account management system 150 transmits a transactionauthorization request to an issuer system. In an example, the accountmanagement system 150 generates a transaction authorization requestcomprising user 101 payment account information, service provider systempayment account information, and a total amount of the transaction. Inan example, the account management system 150 receives the transactionauthorization request from the account management system 150 via thenetwork 140.

In block 1260, the issuer system approves the transaction authorizationrequest and transmits a transaction authorization approval to theaccount management system 150. In an example, the issuer systemsubtracts the user's 101 current account balance from the user's 101credit limit associated with the user's 101 credit account. In anexample, if the different between the credit limit and the creditbalance is a positive number greater than the total amount of thetransaction, the issuer system approves the transaction authorization.For example, the user 101 will not exceed his credit limit if thetransaction were to be approved, therefore the issuer system approvesthe transaction authorization request. In an example, the accountmanagement system 150 receives the transaction authorization approvalfrom the issuer system. In another example, the account managementsystem ISO receives a denial of the transaction authorization request inresponse to the issuer system declining the transaction authorizationrequest. In an example, if the issuer system approves the transactionauthorization request, the issuer system credits the service providersystem account and adds a charge to the user 101 credit accountstatement, bank account statement, or other appropriate statement.

In block 1270, the account management system 150 transmits a transactionreceipt to the service computing device 130. For example, the accountmanagement system 150 transmits a confirmation of the approval of thetransaction or a notice of a declined transaction to the servicecomputing device 130 over the network 140. In an example, the servicecomputing device 130 prints a receipt comprising a summary of asuccessful or unsuccessful transaction via a printer device that is acomponent of or otherwise communicatively coupled to the servicecomputing device 130. In another example, the account management system150 may transmit a transaction receipt to the user computing device 110to be displayed on the user computing device. In yet another example,the service computing device 130 transmits a transaction receipt to theuser computing device 110 to be displayed on the user computing device110.

Additional Examples

In certain examples, the service computing device 130 processes aservice request to determine and/or verify the identity of a user 101for purposes of authentication, granting access, or personalizingexperiences with the user 101 at the service provider system location.In an example, gaining access to a location or authorization to initiatea service request is restricted to a certain type of user 101 based onuser 101 attributes or based on the identity of the user. An exampleuser attribute is the age of the user. In this example, a user 101 signsinto the service application 112 via the user computing device 110 andenters the service provider system location. The user computing device110 receives a service beacon device identifier 120 from the servicebeacon device and transmits the identifier to the account managementsystem 140. The account management system 140 transmits facial templatesto the service computing device 130 corresponding to the user 101 andany other users 101 whose user computing devices 110 are in networkrange of the service computing device, who are signed in to the serviceapplication associated with the account management system. The servicecamera device 130 captures a facial image of the user 101 and identifiesthe user 101 by comparing the captured facial image against the receivedfacial templates to find a match. In addition to or instead of facialrecognition, the service computing device 130 identifies the user 101 ina video feed based on comparing movement data received from the usercomputing device 110 to data from a video feed captured by the servicecamera device 135. In an example, the user 101 requests to purchase arestricted item, for example, an alcoholic beverage. For example, theservice provider system 130 location does not process service requestsfor users that are less than twenty one years old. In this example, theservice computing device 130 approves or denies service requestrequested by the user 101 based on the user 101 identity and/or user 101attributes.

In another example, only users 101 who are employees of the serviceprovider system are authorized to access a door to a restricted room atthe service provider system location. In this example, a user 101attribute comprises a user's 101 name or identity. In this example, theuser 101 requests to access the restricted area of the service providersystem location or otherwise unlock a restricted access door. Forexample, the service computing device 130 comprises an electronic lockor a device communicatively coupled to an electronic lock. In anexample, the service computing device 130 retrieves data comprising aset of user 101 identities allowed access to a restricted area. In anexample, the service computing device 130 unlocks the electronic lock orotherwise allows access to the restricted area if the identity of theuser 101 matches the user 101 identity of one of the users 101 of a setof users 101 allowed access to the restricted area In another example,based on the identity of the user 101 not corresponding to an identifyof one of the set of users 101 allowed access to the restricted area,the service computing device 130 does not allow the user 101 access tothe restricted area.

In another example, the service provider system location provides apersonalized one-on-one experience to a user 101 based on the user's 101service request history and habits. The process involves notifyingservice computing device 130 operators of the entry of a user 101 at theservice provider location, the identity of the user 101, and anyassociated current or past service request history that will allow theservice provider system 130 to streamline the experience and provide asatisfactory experience to the user 101 at the service provider systemlocation. In an example, once the user 101 is identified via movementdata recognition and/or facial recognition, the service computing device130 displays details about the user 101 to the operator of the servicecomputing device 130. For example, the user's 101 name, nickname,interests, and/or other relevant data is displayed to the operator sothat the service computing device 130 operator can provide apersonalized experience to the user at the service provider systemlocation. For example, the operator views the name of the user 101,“Charlotte Thomas,” displayed on the service computing device and says,“How may I help you today, Ms. Thomas?” In this example, the servicecomputing device 130 may display current or past service request historyof the user. For example, the service computing device 130 operatorviews the name of the user 101, “Charlotte Thomas,” and that the user101 requested a particular service request in the past. In this example,the service computing device 130 operator may greet the user 101 asfollows: “How may I help you today, Ms. Thomas?” and then follow up with“Are you satisfied with your previous service request?”

In yet another example, a service provider system may identify a user101 to provide delivery to a user 101. In this example, the user 101signs into the service application via the user computing device 110 andenters the service provider system location. The user computing device110 receives a service beacon device 120 identifier from the servicebeacon device 120 and transmits the identifier to the account managementsystem 140. The account management system 140 transmits facial templatesto the service computing device 130 corresponding to the user 101 andany other users 101 whose user computing devices 110 are in networkrange of the service beacon device 120 and who are signed in to theservice application 112. The service camera device 135 captures a facialimage of the user 101 and identifies the user 101 by comparing thecaptured facial image against the received facial templates to find amatch. In another example, in addition to or instead of identifying theuser 101 via facial recognition, the service computing device 130identifies the user 101 in a video feed based on comparing movement datareceived from the user computing device 110 to data from a video feedcaptured by the service camera device 135. In this example, theidentified user 101 requests to pickup and item that was previouslyordered. In this example, based on the identity of the identified user101 matching a user 101 identity associated with an item for pickup, theoperator of the service computing device 130 delivers the item to theuser 101. For example, the service computing device 130 operatordetermines the item for pickup associated with the identified user 101.

Other Examples

FIG. 13 depicts a computing machine 2000 and a module 2050 in accordancewith certain examples. The computing machine 2000 may correspond to anyof the various computers, servers, mobile devices, embedded systems, orcomputing systems presented herein. The module 2050 may comprise one ormore hardware or software elements configured to facilitate thecomputing machine 2000 in performing the various methods and processingfunctions presented herein. The computing machine 2000 may includevarious internal or attached components such as a processor 2010, systembus 2020, system memory 2030, storage media 2040, input/output interface2060, and a network interface 2070 for communicating with a network2080.

The computing machine 2000 may be implemented as a conventional computersystem, an embedded controller, a laptop, a server, a mobile device, asmartphone, a set-top box, a kiosk, a vehicular information system, onemore processors associated with a television, a customized machine, anyother hardware platform, or any combination or multiplicity thereof. Thecomputing machine 2000 may be a distributed system configured tofunction using multiple computing machines interconnected via a datanetwork or bus system.

The processor 2010 may be configured to execute code or instructions toperform the operations and functionality described herein, managerequest flow and address mappings, and to perform calculations andgenerate commands. The processor 2010 may be configured to monitor andcontrol the operation of the components in the computing machine 2000.The processor 2010 may be a general purpose processor, a processor core,a multiprocessor, a reconfigurable processor, a microcontroller, adigital signal processor (“DSP”), an application specific integratedcircuit (“ASIC”), a graphics processing unit (“GPU”), a fieldprogrammable gate array (“FPGA”), a programmable logic device (“PLD”), acontroller, a state machine, gated logic, discrete hardware components,any other processing unit, or any combination or multiplicity thereof.The processor 2010 may be a single processing unit, multiple processingunits, a single processing core, multiple processing cores, specialpurpose processing cores, co-processors, or any combination thereof.According to certain embodiments, the processor 2010 along with othercomponents of the computing machine 2000 may be a virtualized computingmachine executing within one or more other computing machines.

The system memory 2030 may include non-volatile memories such asread-only memory (“ROM”), programmable read-only memory (“PROM”),erasable programmable read-only memory (“EPROM”), flash memory, or anyother device capable of storing program instructions or data with orwithout applied power. The system memory 2030 may also include volatilememories such as random access memory (“RAM”), static random accessmemory (“SRAM”), dynamic random access memory (“DRAM”), and synchronousdynamic random access memory (“SDRAM”). Other types of RAM also may beused to implement the system memory 2030. The system memory 2030 may beimplemented using a single memory module or multiple memory modules.While the system memory 2030 is depicted as being part of the computingmachine 2000, one skilled in the art will recognize that the systemmemory 2030 may be separate from the computing machine 2000 withoutdeparting from the scope of the subject technology. It should also beappreciated that the system memory 2030 may include, or operate inconjunction with, a non-volatile storage device such as the storagemedia 2040.

The storage media 2040 may include a hard disk, a floppy disk, a compactdisc read only memory (“CD-ROM”), a digital versatile disc (“DVD”), aBlu-ray disc, a magnetic tape, a flash memory, other non-volatile memorydevice, a solid state drive (“SSD”), any magnetic storage device, anyoptical storage device, any electrical storage device, any semiconductorstorage device, any physical-based storage device, any other datastorage device, or any combination or multiplicity thereof. The storagemedia 2040 may store one or more operating systems, application programsand program modules such as module 2050, data, or any other information.The storage media 2040 may be part of, or connected to, the computingmachine 2000. The storage media 2040 may also be part of one or moreother computing machines that are in communication with the computingmachine 2000 such as servers, database servers, cloud storage, networkattached storage, and so forth.

The module 2050 may comprise one or more hardware or software elementsconfigured to facilitate the computing machine 2000 with performing thevarious methods and processing functions presented herein. The module2050 may include one or more sequences of instructions stored assoftware or firmware in association with the system memory 2030, thestorage media 2040, or both. The storage media 2040 may thereforerepresent examples of machine or computer readable media on whichinstructions or code may be stored for execution by the processor 2010.Machine or computer readable media may generally refer to any medium ormedia used to provide instructions to the processor 2010. Such machineor computer readable media associated with the module 2050 may comprisea computer software product. It should be appreciated that a computersoftware product comprising the module 2050 may also be associated withone or more processes or methods for delivering the module 2050 to thecomputing machine 2000 via the network 2080, any signal-bearing medium,or any other communication or delivery technology. The module 2050 mayalso comprise hardware circuits or information for configuring hardwarecircuits such as microcode or configuration information for an FPGA orother PLD.

The input/output (“I/O”) interface 2060 may be configured to couple toone or more external devices, to receive data from the one or moreexternal devices, and to send data to the one or more external devices.Such external devices along with the various internal devices may alsobe known as peripheral devices. The I/O interface 2060 may include bothelectrical and physical connections for operably coupling the variousperipheral devices to the computing machine 2000 or the processor 2010.The I/O interface 2060 may be configured to communicate data, addresses,and control signals between the peripheral devices, the computingmachine 2000, or the processor 2010. The I/O interface 2060 may beconfigured to implement any standard interface, such as small computersystem interface (“SCSI”), serial-attached SCSI (“SAS”), fiber channel,peripheral component interconnect (“PCI”). PCI express (PCe), serialbus, parallel bus, advanced technology attached (“ATA”), serial ATA(“SATA”), universal serial bus (“USB”), Thunderbolt, FireWire, variousvideo buses, and the like. The I/O interface 2060 may be configured toimplement only one interface or bus technology. Alternatively, the I/Ointerface 2060 may be configured to implement multiple interfaces or bustechnologies. The I/O interface 2060 may be configured as part of, allof, or to operate in conjunction with, the system bus 2020. The I/Ointerface 2060 may include one or more buffers for bufferingtransmissions between one or more external devices, internal devices,the computing machine 2000, or the processor 2010.

The I/O interface 2060 may couple the computing machine 2000 to variousinput devices including mice, touch-screens, scanners, electronicdigitizers, sensors, receivers, touchpads, trackballs, cameras,microphones, keyboards, any other pointing devices, or any combinationsthereof. The I/O interface 2060 may couple the computing machine 2000 tovarious output devices including video displays, speakers, printers,projectors, tactile feedback devices, automation control, roboticcomponents, actuators, motors, fans, solenoids, valves, pumps,transmitters, signal emitters, lights, and so forth.

The computing machine 2000 may operate in a networked environment usinglogical connections through the network interface 2070 to one or moreother systems or computing machines across the network 2080. The network2080 may include wide area networks (WAN), local area networks (LAN),intranets, the Internet, wireless access networks, wired networks,mobile networks, telephone networks, optical networks, or combinationsthereof. The network 2080 may be packet switched, circuit switched, ofany topology, and may use any communication protocol. Communicationlinks within the network 2080 may involve various digital or an analogcommunication media such as fiber optic cables, free-space optics,waveguides, electrical conductors, wireless links, antennas,radio-frequency communications, and so forth.

The processor 2010 may be connected to the other elements of thecomputing machine 2000 or the various peripherals discussed hereinthrough the system bus 2020. It should be appreciated that the systembus 2020 may be within the processor 2010, outside the processor 2010,or both. According to some embodiments, any of the processor 2010, theother elements of the computing machine 2000, or the various peripheralsdiscussed herein may be integrated into a single device such as a systemon chip (“SOC”), system on package (“SOP”), or ASIC device.

In situations in which the systems discussed here collect personalinformation about users, or may make use of personal information, theusers may be provided with an opportunity or option to control whetherprograms or features collect user information (e.g., information about auser's social network, social actions or activities, profession, auser's preferences, or a user's current location), or to control whetherand/or how to receive content from the content server that may be morerelevant to the user. In addition, certain data may be treated in one ormore ways before it is stored or used, so that personally identifiableinformation is removed. For example, a user's identity may be treated sothat no personally identifiable information can be determined for theuser, or a user's geographic location may be generalized where locationinformation is obtained (such as to a city, ZIP code, or state level),so that a particular location of a user cannot be determined. Thus, theuser may have control over how information is collected about the userand used by a content server.

Embodiments may comprise a computer program that embodies the functionsdescribed and illustrated herein, wherein the computer program isimplemented in a computer system that comprises instructions stored in amachine-readable medium and a processor that executes the instructions.However, it should be apparent that there could be many different waysof implementing embodiments in computer programming, and the embodimentsshould not be construed as limited to any one set of computer programinstructions. Further, a skilled programmer would be able to write sucha computer program to implement an embodiment of the disclosedembodiments based on the appended flow charts and associated descriptionin the application text. Therefore, disclosure of a particular set ofprogram code instructions is not considered necessary for an adequateunderstanding of how to make and use embodiments. Further, those skilledin the art will appreciate that one or more aspects of embodimentsdescribed herein may be performed by hardware, software, or acombination thereof, as may be embodied in one or more computingsystems. Moreover, any reference to an act being performed by a computershould not be construed as being performed by a single computer as morethan one computer may perform the act.

The examples described herein can be used with computer hardware andsoftware that perform the methods and processing functions describedherein. The systems, methods, and procedures described herein can beembodied in a programmable computer, computer-executable software, ordigital circuitry. The software can be stored on computer-readablemedia. For example, computer-readable media can include a floppy disk,RAM, ROM, hard disk, removable media, flash memory, memory stick,optical media, magneto-optical media, CD-ROM, etc. Digital circuitry caninclude integrated circuits, gate arrays, building block logic, fieldprogrammable gate arrays (FPGA), etc.

The example systems, methods, and acts described in the embodimentspresented previously are illustrative, and, in alternative embodiments,certain acts can be performed in a different order, in parallel with oneanother, omitted entirely, and/or combined between different examples,and/or certain additional acts can be performed, without departing fromthe scope and spirit of various embodiments. Accordingly, suchalternative embodiments are included in the scope of the followingclaims, which are to be accorded the broadest interpretation so as toencompass such alternate embodiments.

Although specific embodiments have been described above in detail, thedescription is merely for purposes of illustration. It should beappreciated, therefore, that many aspects described above are notintended as required or essential elements unless explicitly statedotherwise. Modifications of, and equivalent components or actscorresponding to, the disclosed aspects of the examples, in addition tothose described above, can be made by a person of ordinary skill in theart, having the benefit of the present disclosure, without departingfrom the spirit and scope of embodiments defined in the followingclaims, the scope of which is to be accorded the broadest interpretationso as to encompass such modifications and equivalent structures.

1-20. (canceled)
 21. A computer-implemented method to identify users,comprising, by one or more processors: obtaining visual data associatedwith one or more images depicting an object during a time period;obtaining movement data associated with a user computing device duringthe time period; providing the visual data and the movement data to oneor more machine-learned classification models configured to identifysimilarities between the visual data and the movement data; comparing,using the one or more machine-learned classification models, the visualdata and the movement data to determine a correlation during the timeperiod between the object depicted in the one or more images of thevisual data and the user computing device associated with the movementdata; and based on a determined correlation between the visual data andthe movement data, identifying the object depicted in the one or moreimages as corresponding to a user of the user computing device.
 22. Thecomputer-implemented method of claim 21, further comprising, by the oneor more processors: detecting the object from a video feed comprisingthe visual data associated with the one or more images; and determininga change in position of the object in the video feed during the timeperiod; wherein the visual data provided to the one or moremachine-learned classification models includes data indicative of thechange in position of the object in the video feed.
 23. Thecomputer-implemented method of claim 22, wherein comparing the visualdata and the movement data to determine the correlation comprises:determining the correlation during the time period based on the changein the position of the object in the video feed and the movement dataassociated with the user computing device.
 24. The computer-implementedmethod of claim 22, wherein determining the change in the position ofthe object in the video feed comprises determining a movement signatureof the object.
 25. The computer-implemented method of claim 22, whereinthe movement data comprises a gyroscopic signature, wherein determiningthe change in the position of the object in the video feed comprisesdetermining a movement signature of the object, and wherein comparingthe change in the position of the object in the video feed with thechange in the movement data received from the user computing device todetermine the correlation comprises comparing the gyroscopic signatureto the movement signature.
 26. The computer-implemented method of claim21, wherein: the user computing device is a particular user computingdevice: obtaining movement data comprises receiving, from each of aplurality of user computing devices, respective movement data indicatingmovement of such user computer device during the time period; providingthe movement data to the one or more classification machine-learnedmodels comprises providing the respective movement data for each usercomputing device to the one or more machine-learned classificationmodels; and comparing the visual data and the movement data comprisesdetermining the correlation during the time period between the objectdepicted in the one or more images of the visual data and the particularuser computing device based on the respective movement data indicatingmovement of the particular user computing device.
 27. Thecomputer-implemented method of claim 21, wherein the movement datacomprises at least one of a gyroscopic signature or an accelerometersignature.
 28. The computer-implemented method of claim 21, furthercomprising, by the one or more processors: receiving a request toinitiate a service request; and transmitting a user computing deviceidentifier corresponding to the user computing device and a request toprocess the service request to an account management computing system.29. The computer-implemented method of claim 28, further comprising, bythe one or more processors: receiving, from the account managementcomputing system, data corresponding to one or more accounts of theparticular user; displaying at least a portion of the data correspondingto the one or more accounts of the particular user; receiving an inputof a selection of a particular account for use in the service request;and transmitting, to the account management computing system, theselection of the particular account, wherein the account managementcomputing system processes the service request using the particularaccount.
 30. The computer-implemented method of claim 28, furthercomprising, by the one or more processors: determining a change inposition of the object from a first position to a second position duringthe time period determining that the second position of the object iswithin a defined area of a video feed associated with a service requestarea, wherein the request to process the service request is transmittedin response to determining that the second position of the object iswithin the service request area associated with the video feed.
 31. Acomputing system, comprising: one or more processors; and one ornon-transitory computer readable media that collectively storeinstructions that when executed by the one or more processors cause theone or more processors to perform operations, the operations comprising:obtaining visual data associated with one or more images depicting anobject during a time period; obtaining movement data associated with auser computing device during the time period; providing the visual dataand the movement data to one or more machine-learned classificationmodels configured to identify similarities between the visual data andthe movement data; comparing, using the one or more machine-learnedclassification models, the visual data and the movement data todetermine a correlation during the time period between the objectdepicted in the one or more images of the visual data and the usercomputing device associated with the movement data; and based on adetermined correlation between the visual data and the movement data,identifying the object depicted in the one or more images ascorresponding to a user of the user computing device.
 32. The computingsystem of claim 31, wherein the operations further comprise: detectingthe object from a video feed comprising the visual data associated withthe one or more images; and determining a change in position of theobject in the video feed during the time period; wherein the visual dataprovided to the one or more machine-learned classification modelsincludes data indicative of the change in position of the object in thevideo feed.
 33. The computing system of claim 32, wherein comparing thevisual data and the movement data to determine the correlationcomprises: determining the correlation during the time period based onthe change in the position of the object in the video feed and themovement data associated with the user computing device.
 34. Thecomputing system of claim 32, wherein determining the change in theposition of the object in the video feed comprises determining amovement signature of the object.
 35. The computing system of claim 32,wherein the movement data comprises a gyroscopic signature, whereindetermining the change in the position of the object in the video feedcomprises determining a movement signature of the object, and whereincomparing the change in the position of the object in the video feedwith the change in the movement data received from the user computingdevice to determine the correlation comprises comparing the gyroscopicsignature to the movement signature.
 36. The computing system of claim31, wherein: the user computing device is a particular user computingdevice: obtaining movement data comprises receiving, from each of aplurality of user computing devices, respective movement data indicatingmovement of such user computer device during the time period; providingthe movement data to the one or more classification machine-learnedmodels comprises providing the respective movement data for each usercomputing device to the one or more machine-learned classificationmodels; and comparing the visual data and the movement data comprisesdetermining the correlation during the time period between the objectdepicted in the one or more images of the visual data and the particularuser computing device based on the respective movement data indicatingmovement of the particular user computing device.
 37. One or morenon-transitory computer readable media that collectively storeinstructions that when executed by one or more processors cause the oneor more processors to perform operations, the operations comprising:obtaining visual data associated with one or more images depicting anobject during a time period; obtaining movement data associated with auser computing device during the time period; providing the visual dataand the movement data to one or more machine-learned classificationmodels configured to identify similarities between the visual data andthe movement data; comparing, using the one or more machine-learnedclassification models, the visual data and the movement data todetermine a correlation during the time period between the objectdepicted in the one or more images of the visual data and the usercomputing device associated with the movement data; and based on adetermined correlation between the visual data and the movement data,identifying the object depicted in the one or more images ascorresponding to a user of the user computing device.
 38. The one ormore non-transitory computer readable media of claim 37, wherein theoperations further comprise: receiving a request to initiate a servicerequest; and transmitting a user computing device identifiercorresponding to the user computing device and a request to process theservice request to an account management computing system.
 39. The oneor more non-transitory computer readable media of claim 38, furthercomprising, wherein the operations further comprise: receiving, from theaccount management computing system, data corresponding to one or moreaccounts of the particular user; displaying at least a portion of thedata corresponding to the one or more accounts of the particular user;receiving an input of a selection of a particular account for use in theservice request; and transmitting, to the account management computingsystem, the selection of the particular account, wherein the accountmanagement computing system processes the service request using theparticular account.
 40. The one or more non-transitory computer readablemedia of claim 38, further comprising, wherein the operations furthercomprise: determining a change in position of the object from a firstposition to a second position during the time period determining thatthe second position of the object is within a defined area of a videofeed associated with a service request area, wherein the request toprocess the service request is transmitted in response to determiningthat the second position of the object is within the service requestarea associated with the video feed.